PRE-SETUP = We are simply getting the environment and data ready
TRAIN NEW H2O MODEL = To make use of H2O’s libraries, we need to create the models in H2O (ie. our original RF model cannot be directly converted to an h2o model)
COMPARE MODEL’S PREDICTIONS = We have the OrigRF and all the other new models generate predictions on the testing data
COMPARE MODEL’S METRICS = A true validation of the models’ performance, and a correct way of validating if our new models make good proxies for the original model
LIME = Here we actually begin to apply LIME to our RF models
SHAP Values = We use the H2O library to generate the SHAP values for our H2O RF model
PDPs = Similar to the previous section, we use the H2O library to generate the PDPs for the H2O RF model

Introduction

The goal of this project was to apply interpretability models to the random forest classifier in the PeerJ article “Prioritizing bona fide bacterial small RNAs with machine learning classifiers” with the purposes of gaining deeper insights into the prioritization of bona fide bacterial sRNAs.

Some key Terms

OrigRF/Orig RF = Original random forest as found in link
OrigRF_[scaled/normalized] = refers to any model that was trained in the same manner(same libraries and settings) as in the Original random forest model, but using scaled or normalized data.
- By scaling the data we mean scaling and centering
- By normalizing we mean fit the values between 0 and 1
RFE = Random Forest Explainer
RF = Random Forest
IM = Interpretability Model
“SS” = the free energy of the predicted secondary structure of the sRNA - mostly negative values
“Pos10wrtsRNAStart” = distance to their closest predicted promoter site
“DistTerm” = distance to their closest predicted Rho-independent terminator
“Distance” = distance to the closest reading frame on the LEFT(“upstream”) side
“sameStrand” = boolean, if transcription is going in the same direction as the ORF(Left Open Reading Frame)
- ORF = genomic sequence that’s supposed to code for a protein
“DownDistance” = distance to the closest reading frame on the RIGHT(“downstream”) side
“sameDownStrand” = boolean, if transcription is going in the same direction as the RORF(Right Open Reading Frame)

Side note:* When used the word original to refer to the RF model created in the base article, or to refer to models that were created(trained) in the same way as the base random forest model.

I) PRE-SETUP

This section is all about getting the libraries and data needed to run this R notebook. We will preprocess the data and load the original model. We start with some OPTIONAL memory cleaning (useful if you want to run this from R Studio and with a clean environment)

# This section is just about getting the environment ready and cleaned up. Since this project was originally written as a regular R script, and worked from within R Studio, it was just usefull in resetting and cleaning up the the environment, and so this block of code is optional
clc <- function() cat("\014") ; clc()

remove(list = ls())
gc()

          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 2479304 132.5    6565627 350.7  6565627 350.7
Vcells 5037059  38.5   26355833 201.1 65162034 497.2

1. Install required libraries

The following code is based on the h2o documentation site, as it’s their recommended way for downloading and installing h2o for R, with only the first if being added as to prevent accidental reinstallations. Here is the link for those interested in it.

if (! ("h2o" %in% rownames(installed.packages()))) { 
  
    # The following two commands remove any previously installed H2O packages for R.
    if ("package:h2o" %in% search()) { detach("package:h2o", unload=TRUE) }
    if ("h2o" %in% rownames(installed.packages())) { remove.packages("h2o") }
    # Next, we download packages that H2O depends on.
    pkgs <- c("RCurl","jsonlite")
    for (pkg in pkgs) {
      if (! (pkg %in% rownames(installed.packages()))) { install.packages(pkg) }
    }
    # Now we download, install and initialize the H2O package for R.
    install.packages("h2o", type="source", repos="http://h2o-release.s3.amazonaws.com/h2o/rel-zahradnik/4/R")
}

The rest of the libraries can be installed in the same way as with any other library.

packages <- c("lime", "ROCR", "PRROC", "randomForest", "randomForestExplainer", "png")
for (package in packages) {
  if (! (package %in% rownames(installed.packages()))) { install.packages(package) }
}

2. Load the libraries

library("h2o")          # ML model building
library("ROCR")         # ML evaluation
library("PRROC")        # ML evaluation
library("randomForest") # ML model building
library("randomForestExplainer") # ML Global interpretation
library("lime")         # ML local interpretation
library("png")

3. Load and preprocess the data

3.1 Load the Model’s Training Data

trainDataSet <- read.csv("./DataSets/combinedData.csv", header = TRUE)
trainDataSet[,"Class"] <- as.logical(trainDataSet[,"Class"]) # guarantees that our h2o model trains the random forest (RF) as a classification tree and not a regression tree

3.2 Scale and Center the Training Data

dataSetTrain_scaled <- scale(trainDataSet[,-c(5,7,8,9)], center = TRUE, scale = TRUE)
dataSetTrain_scaled <- cbind( dataSetTrain_scaled[,(1:4)], 
                              trainDataSet[,5],
                              dataSetTrain_scaled[,5],
                              trainDataSet[,c(7,8,9)])
colnames(dataSetTrain_scaled) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance", 
                                   "sameStrand", "DownDistance", "sameDownStrand", "ID", "Class")
dataSetTrain_scaled[,"Class"] <- as.logical(dataSetTrain_scaled[,"Class"]) # Just making sure Class is taken a logical value
rbind( head(dataSetTrain_scaled,5), tail(dataSetTrain_scaled,5) )

3.3 Normalize the Training Data

normalize <- function(x) {
  return ((x - min(x)) / (max(x) - min(x)))
}
dataSetTrain_norm <- trainDataSet
dataSetTrain_norm$SS <- normalize(dataSetTrain_norm$SS)
dataSetTrain_norm$Pos10wrtsRNAStart <- normalize(dataSetTrain_norm$Pos10wrtsRNAStart)
dataSetTrain_norm$DistTerm <- normalize(dataSetTrain_norm$DistTerm)
dataSetTrain_norm$Distance <- normalize(dataSetTrain_norm$Distance)
dataSetTrain_norm$DownDistance <- normalize(dataSetTrain_norm$DownDistance)
dataSetTrain_norm[,"Class"] <- as.logical(dataSetTrain_norm[,"Class"]) # Just making sure Class is taken a logical value
rbind( head(dataSetTrain_norm,5), tail(dataSetTrain_norm,5) )

3.4 Check for normality in the data

There are certain parameter() in the LIME functions that are recommended based on the distribution of the data. Based on these paremeters, we decided to check for normality in the data

dataSetTrainX <- trainDataSet[,-(8:9)]
dataSetTrainY <- trainDataSet[,(8:9)]  
hist(x = as.numeric(dataSetTrainX[,1]), main = "Histogram of SS")

qqnorm(as.numeric(dataSetTrainX[,1]))
qqline(as.numeric(dataSetTrainX[,1]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,2]), main = "Histogram of Pos10wrtsRNAStart")

qqnorm(as.numeric(dataSetTrainX[,2]))
qqline(as.numeric(dataSetTrainX[,2]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,3]), main = "Histogram of DistTerm")

qqnorm(as.numeric(dataSetTrainX[,3]))
qqline(as.numeric(dataSetTrainX[,3]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,4]), main = "Histogram of Distance")

qqnorm(as.numeric(dataSetTrainX[,4]))
qqline(as.numeric(dataSetTrainX[,4]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,5]), main = "Histogram of sameStrand")

qqnorm(as.numeric(dataSetTrainX[,5]))
qqline(as.numeric(dataSetTrainX[,5]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,6]), main = "Histogram of DownDistance")

qqnorm(as.numeric(dataSetTrainX[,6]))
qqline(as.numeric(dataSetTrainX[,6]), col = "red", lwd = 2)

hist(x = as.numeric(dataSetTrainX[,7]), main = "Histogram of sameDownStrand")

qqnorm(as.numeric(dataSetTrainX[,7]))
qqline(as.numeric(dataSetTrainX[,7]), col = "red", lwd = 2)

3.5 Load testing Datasets

Load Original Testing Data Sets

slt2dataPos <- read.csv("./DataSets/SLT2_Positives.tsv", sep = "\t", header = TRUE)
slt2dataPos$Class <- rep(1,nrow(slt2dataPos))
slt2dataNeg <- read.csv("./DataSets/SLT2_Negatives.tsv", sep = "\t", header = TRUE)
slt2dataNeg$Class <- rep(0,nrow(slt2dataNeg))
slt2data <- rbind(slt2dataPos,slt2dataNeg)
ludataPos <- read.csv("./DataSets/Lu_Positives.tsv", sep = "\t", header = TRUE)
ludataPos$Class <- rep(1,nrow(ludataPos))
ludataNeg <- read.csv("./DataSets/Lu_Negatives.tsv", sep = "\t", header = TRUE)
ludataNeg$Class <- rep(0,nrow(ludataNeg))
ludata <- rbind(ludataPos,ludataNeg)

3.6 Scale testing Datasets

Even though the scale function can scale and center the data for us, we need to scale the testing data sets using the training data’s standard deviation and means to obtain adequate results.

Scaling Function: * Xi = An instance * Xmean = Mean/Average of all samples
* Xsd = Standard Deviation of our data set * Zi = Scaled Value of Xi * Zi = (Xi - Xmean) / Xsd

# Standard Deviation and Mean for Standarization
scale_td <- function( Xi, Xmean, Xsd) {
  return ( (Xi - Xmean) / Xsd ) # Zi
}
trainData_sd_SS        <- sd(trainDataSet[,"SS"])
trainData_sd_Pos       <- sd(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_sd_DisTerm   <- sd(trainDataSet[,"DistTerm"])
trainData_sd_Dis       <- sd(trainDataSet[,"Distance"])
trainData_sd_DownDis   <- sd(trainDataSet[,"DownDistance"])
trainData_mean_SS      <- mean(trainDataSet[,"SS"])
trainData_mean_Pos     <- mean(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_mean_DisTerm <- mean(trainDataSet[,"DistTerm"])
trainData_mean_Dis     <- mean(trainDataSet[,"Distance"])
trainData_mean_DownDis <- mean(trainDataSet[,"DownDistance"])
slt2data_scaled <- slt2data
slt2data_scaled$SS                <- scale_td(slt2data_scaled$SS, 
                                              trainData_mean_SS, trainData_sd_SS)
slt2data_scaled$Pos10wrtsRNAStart <- scale_td(slt2data_scaled$Pos10wrtsRNAStart,
                                              trainData_mean_Pos, trainData_sd_Pos)
slt2data_scaled$DistTerm          <- scale_td(slt2data_scaled$DistTerm, 
                                              trainData_mean_DisTerm, trainData_sd_DisTerm)
slt2data_scaled$Distance          <- scale_td(slt2data_scaled$Distance,
                                              trainData_mean_Dis, trainData_sd_Dis)
slt2data_scaled$DownDistance      <- scale_td(slt2data_scaled$DownDistance,
                                              trainData_mean_DownDis, trainData_sd_DownDis)
ludata_scaled <- ludata
ludata_scaled$SS                <- scale_td(ludata_scaled$SS, 
                                              trainData_mean_SS, trainData_sd_SS)
ludata_scaled$Pos10wrtsRNAStart <- scale_td(ludata_scaled$Pos10wrtsRNAStart,
                                              trainData_mean_Pos, trainData_sd_Pos)
ludata_scaled$DistTerm          <- scale_td(ludata_scaled$DistTerm, 
                                              trainData_mean_DisTerm, trainData_sd_DisTerm)
ludata_scaled$Distance          <- scale_td(ludata_scaled$Distance,
                                              trainData_mean_Dis, trainData_sd_Dis)
ludata_scaled$DownDistance      <- scale_td(ludata_scaled$DownDistance,
                                              trainData_mean_DownDis, trainData_sd_DownDis)
rbind( head(slt2data_scaled,5), tail(slt2data_scaled,5))

rbind( head(ludata_scaled,5), tail(ludata_scaled,5))

3.7 Normalize testing Datasets

In a similar fashion to what we did in 3.6, we also need to use the training data’s max and min values to properly normalize each feature.

normalize_td <- function(x, min, max) {
  return ( (x - min) / (max - min) )
}
trainData_max_SS      <- max(trainDataSet[,"SS"])
trainData_max_Pos     <- max(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_max_DisTerm <- max(trainDataSet[,"DistTerm"])
trainData_max_Dis     <- max(trainDataSet[,"Distance"])
trainData_max_DownDis <- max(trainDataSet[,"DownDistance"])
trainData_min_SS      <- min(trainDataSet[,"SS"])
trainData_min_Pos     <- min(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_min_DisTerm <- min(trainDataSet[,"DistTerm"])
trainData_min_Dis     <- min(trainDataSet[,"Distance"])
trainData_min_DownDis <- min(trainDataSet[,"DownDistance"])
slt2data_norm <- slt2data
slt2data_norm$SS                <- normalize_td(slt2data_norm$SS, 
                                                 trainData_min_SS, trainData_max_SS)
slt2data_norm$Pos10wrtsRNAStart <- normalize_td(slt2data_norm$Pos10wrtsRNAStart,
                                                 trainData_min_Pos, trainData_max_Pos)
slt2data_norm$DistTerm          <- normalize_td(slt2data_norm$DistTerm,
                                        trainData_min_DisTerm, trainData_max_DisTerm)
slt2data_norm$Distance          <- normalize_td(slt2data_norm$Distance,
                                        trainData_min_Dis, trainData_max_Dis)
slt2data_norm$DownDistance      <- normalize_td(slt2data_norm$DownDistance,
                                            trainData_min_DownDis, trainData_max_DownDis)
ludata_norm <- ludata
ludata_norm$SS                <- normalize_td(ludata_norm$SS, 
                                                 trainData_min_SS, trainData_max_SS)
ludata_norm$Pos10wrtsRNAStart <- normalize_td(ludata_norm$Pos10wrtsRNAStart,
                                                 trainData_min_Pos, trainData_max_Pos)
ludata_norm$DistTerm          <- normalize_td(ludata_norm$DistTerm,
                                                 trainData_min_DisTerm, trainData_max_DisTerm)
ludata_norm$Distance          <- normalize_td(ludata_norm$Distance,
                                                 trainData_min_Dis, trainData_max_Dis)
ludata_norm$DownDistance      <- normalize_td(ludata_norm$DownDistance,
                                                 trainData_min_DownDis, trainData_max_DownDis)
rbind( head(slt2data_norm,5), tail(slt2data_norm,5))

rbind( head(ludata_norm,5), tail(ludata_norm,5))

4. Load original model

The original article, titled “Prioritizing bona fide bacterial small RNAs with machine learning classifiers”, and the source materials(training data, testing data, R scripts, .rds file, etc.) can all be found in PeerJ under the following link. The original .rds file can also be downloaded from github.

4.1 Load/Retrain Orig Model

We can either retrain the original model, or load the .rds file provided in the original article. For the purpose of saving time, we simply loaded the model.

origRF <- readRDS("RF_classifier4sRNA.rds")

4.2 Retrain new models with Scaled and Normalized Data

Since some of the IM models required scaling or normalized data to properly function, we needed to create 2 new models in the same way as the one loaded in 4.1.

tuneRF(dataSetTrainX[,c(1:7)], y = factor(dataSetTrainY[,2]), ntreeTry = 400, mtryStart = 2)

mtry = 2  OOB error = 10.74% 
Searching left ...
mtry = 1    OOB error = 11.5% 
-0.07142857 0.05 
Searching right ...
mtry = 4    OOB error = 10.43% 
0.02857143 0.05 
      mtry  OOBError
1.OOB    1 0.1150307
2.OOB    2 0.1073620
4.OOB    4 0.1042945

set.seed(1234)
origRF_scaled <- randomForest(x = dataSetTrain_scaled[,c(1:7)], y = factor(dataSetTrainY[,2]), mtry = 2, ntree = 400, importance = TRUE)
origRF_norm <- randomForest(x = dataSetTrain_norm[,c(1:7)], y = factor(dataSetTrainY[,2]), mtry = 2, ntree = 400, importance = TRUE)

II) TRAIN THE NEW H2O MODEL

Since we are going to use the h2o library to perform some of our machine learning interpretability, we need to train an equivalent one, using the h2o tools.

5. Intialize H2O and load training data

h2o.init(              # Initialize h2o
  nthreads = -1,       # -1 = use all available threads
  max_mem_size = "8G"  # specify the amount of memory to use
)

 Connection successful!

R is connected to the H2O cluster: 
    H2O cluster uptime:         3 days 10 hours 
    H2O cluster timezone:       America/St_Johns 
    H2O data parsing timezone:  UTC 
    H2O cluster version:        3.30.0.4 
    H2O cluster version age:    1 month and 4 days  
    H2O cluster name:           H2O_started_from_R_carlos.salcedo_lix921 
    H2O cluster total nodes:    1 
    H2O cluster total memory:   6.82 GB 
    H2O cluster total cores:    8 
    H2O cluster allowed cores:  8 
    H2O cluster healthy:        TRUE 
    H2O Connection ip:          localhost 
    H2O Connection port:        54321 
    H2O Connection proxy:       NA 
    H2O Internal Security:      FALSE 
    H2O API Extensions:         Amazon S3, XGBoost, Algos, AutoML, Core V3, TargetEncoder, Core V4 
    R Version:                  R version 3.6.1 (2019-07-05)

h2o.removeAll()        # clean up the system, h2o-wise (ie. kill any running h2o clusters)
h2o.no_progress() 
trainData <- as.h2o(trainDataSet) 
trainData_scaled <- as.h2o(dataSetTrain_scaled) 
trainData_norm <- as.h2o(dataSetTrain_norm)

6. Build model

6.1 Build model with Orig Training data

rfh2o <- h2o.randomForest( 
             training_frame = trainData, # training using the normal data
             x = 1:7,                    # features to use to generate the prediction
             y = 9,                      # Class type -> what we want to predict
             model_id = "rf1_sRNA",      # name of model in h2o
             ntrees = 400,               # max number of trees  
             seed = 1234,                # seed, has to be set WITHIN the h2o function and it's supposed to be different from "R's seed", so results might not be exactly the same as orig model, but should be similar enough
             mtries = 2,                 # Same as original model 
             max_depth = 30
)

6.2 Build model with Scaled Training data

rfh2o_scaled <- h2o.randomForest( 
  training_frame = trainData_scaled,
  x = 1:7,                
  y = 9,                     
  model_id = "rf2_sRNA",     
  ntrees = 400,              
  seed = 1234,              
  mtries = 2,                
  max_depth = 30
)

6.3 Build model with Norm Training data

rfh2o_norm <- h2o.randomForest( 
  training_frame = trainData_norm, 
  x = 1:7,                  
  y = 9,                    
  model_id = "rf3_sRNA",     
  ntrees = 400,               
  seed = 1234,               
  mtries = 2,                  
  max_depth = 30
)

6.4 BONUS: Build GLM model

As we were doing tests with LIME, we discovered that it yielded irregular responses, and as a result, the explanations could not be trusted at the time. It seems to be a known issue that LIME is not perfect for every model, and that it struggles with data that is highly diverse. In this use case, our data contains mixed numerical and categorical features, significantly different scales and magnitudes in our numercial features, and an inbalanced data sets that favors predictions of sRNAs being false. Since lime creates a local interpretable linear model for its explanations, the GLM constructed here was simply to verify the hypothesis that a linear model would be unable to explain the data. The “unbalanced-ness” seems to create models where the features tend always support a result of “false”, while claiming that all the features are against a result of true, which will be evicenced in section 15.1.

glmh2o <- h2o.glm( # https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/glm.html 
  training_frame = trainData, # training using the normal data
  x = 1:7,                   # features to use to generate the prediction
  y = 9,                     # Class type -> what we want to predict
  model_id = "glm_sRNA",     # name of model in h2o
  seed = 1234,
  family = "binomial"
)
glmh2o_perf <- h2o.performance(glmh2o)
glmh2o_perf

H2OBinomialMetrics: glm
** Reported on training data. **

MSE:  0.1476104
RMSE:  0.3842009
LogLoss:  0.4680047
Mean Per-Class Error:  0.2607362
AUC:  0.7639542
AUCPR:  0.5749941
Gini:  0.5279085
R^2:  0.2127447
Residual Deviance:  610.2781
AIC:  626.2781

Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:


Maximum Metrics: Maximum metrics at their respective thresholds


Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`

h2o.coef(glmh2o)

        Intercept                SS Pos10wrtsRNAStart          DistTerm          Distance        sameStrand      DownDistance    sameDownStrand 
    -5.588791e-02     -1.618068e-02      4.486513e-05     -2.304662e-03     -5.368581e-05     -4.228112e-01      1.958833e-04      2.178087e-01

h2o.coef_norm(glmh2o)

        Intercept                SS Pos10wrtsRNAStart          DistTerm          Distance        sameStrand      DownDistance    sameDownStrand 
      -1.31299148       -0.37876153        0.01831303       -0.93619002       -0.02021210       -0.21155895        0.08702437        0.10892590

glmh2o@model$coefficients_table

Coefficients: glm coefficients

h2o.r2(glmh2o) # R^2 value is really low, and having a Max Recall being really low, meaning that LIME is also predicting that everything should be false

[1] 0.2127447

7. Preview H2O’s RF

This is the performance with the OOB error, based on the training process (ie training data)

rfh2o

Model Details:
==============

H2OBinomialModel: drf
Model ID:  rf1_sRNA 
Model Summary:



H2OBinomialMetrics: drf
** Reported on training data. **
** Metrics reported on Out-Of-Bag training samples **

MSE:  0.08046297
RMSE:  0.28366
LogLoss:  0.2701744
Mean Per-Class Error:  0.1247444
AUC:  0.9370883
AUCPR:  0.8592391
Gini:  0.8741767
R^2:  0.5708641

Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:


Maximum Metrics: Maximum metrics at their respective thresholds


Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`

rfh2o@model$variable_importances

Variable Importances:

h2o.varimp_plot(rfh2o)

rfh2o_training_peformance <- h2o.performance(rfh2o)
rfh2o_training_peformance

H2OBinomialMetrics: drf
** Reported on training data. **
** Metrics reported on Out-Of-Bag training samples **

MSE:  0.08046297
RMSE:  0.28366
LogLoss:  0.2701744
Mean Per-Class Error:  0.1247444
AUC:  0.9370883
AUCPR:  0.8592391
Gini:  0.8741767
R^2:  0.5708641

Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:


Maximum Metrics: Maximum metrics at their respective thresholds


Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`

III) COMPARE MODEL’S PREDICTIONS

In this section we’ll get the predictions all the models generate on the testing data LU and SLT2. The goal of this section is not to prove which model is the best, but to prove that the models are similar enough to the point that the analysis of the H2O model will be a valid analogy model for the original RF model.

8. Get Testing Data Predictions from the Models

origRF_lu_pred <- predict(origRF, ludata[,-8], type = "prob")
origRF_slt2_pred <- predict(origRF, slt2data[,-8], type = "prob")
origRF_scaled_lu_pred <- predict(origRF_scaled, ludata_scaled[,-8], type = "prob")
origRF_scaled_slt2_pred <- predict(origRF_scaled, slt2data_scaled[,-8], type = "prob")
origRF_norm_lu_pred <- predict(origRF_norm, ludata_norm[,-8], type = "prob")
origRF_norm_slt2_pred <- predict(origRF_norm, slt2data_norm[,-8], type = "prob")
rfh2o_slt2_pred <- h2o.predict(object = rfh2o, newdata = as.h2o(slt2data[,-8]) )
rfh2o_lu_pred <- h2o.predict(object = rfh2o, newdata = as.h2o(ludata[,-8]) )
rfh2o_scaled_slt2_pred <- h2o.predict(object = rfh2o_scaled, newdata = as.h2o(slt2data_scaled[,-8]) )
rfh2o_scaled_lu_pred <- h2o.predict(object = rfh2o_scaled, newdata = as.h2o(ludata_scaled[,-8]) )
rfh2o_norm_slt2_pred <- h2o.predict(object = rfh2o_norm, newdata = as.h2o(slt2data_norm[,-8]) )
rfh2o_norm_lu_pred <- h2o.predict(object = rfh2o_norm, newdata = as.h2o(ludata_norm[,-8]) )
glm_slt2_pred <- h2o.predict(object = glmh2o, newdata = as.h2o(slt2data[,-8]) )
glm_lu_pred <- h2o.predict(object = glmh2o, newdata = as.h2o(ludata[,-8]) )

9. Create a comparison function

This function is to simplify the comparison between the predictions from the Original RF model and the H2O RF model. Later on this, this function will be used to compare the predictions between the OrigRF model and the other models.

# This function will be used later on, and 
#   a = the original model's predictions
#   b = the alternate model's predictions
#   c = the correct prediction
compareAnswers <- function(a,b,c){
  if ( a == c & b == c ) 
      { res="BOTH_RIGHT" }
  else if ( a == c & b != c ) 
      { res="OnlyA"  }
  else if ( a != c & b == c ) 
      { res="OnlyB"  }
  else 
      { res="BOTH_WRONG"  }
  return(res)
}

10. SLT2 comparisons

10.1 Build SLT2 Predictions table

slt2_predictions <-  cbind(as.data.frame(slt2data),               # Input
                           as.data.frame(origRF_slt2_pred[,2]),   # Orig RF predictions 
                           as.data.frame(origRF_scaled_slt2_pred[,2]),   # Orig RF Scaled predictions 
                           as.data.frame(origRF_norm_slt2_pred[,2]),   # Orig RF Normalized predictions   
                           as.data.frame(rfh2o_slt2_pred[,3]),        # H2O RF predictions
                           as.data.frame(rfh2o_scaled_slt2_pred[,3]), # Scaled RF predictions
                           as.data.frame(rfh2o_norm_slt2_pred[,3])    # Normalized RF predictions
                           )
colnames(slt2_predictions) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance", 
                                "sameStrand", "DownDistance", "sameDownStrand", "Class",
                                "OrigRF_P","OrigRF_scaled_P","OrigRF_norm_P",
                                "RF_H2O_P","RFH2O_scaled_P","RFH2O_norm_P")
slt2_predictions$origPreds <- ifelse( slt2_predictions$OrigRF_P >= 0.5, 1, 0) 
slt2_predictions$origScaledPreds <- ifelse( slt2_predictions$OrigRF_scaled_P >= 0.5, 1, 0) 
slt2_predictions$origNormPreds <- ifelse( slt2_predictions$OrigRF_norm_P >= 0.5, 1, 0) 
slt2_predictions$h2oPreds  <- ifelse( slt2_predictions$RF_H2O_P >= 0.5, 1, 0) 
slt2_predictions$h2oScaledPreds  <- ifelse( slt2_predictions$RFH2O_scaled_P >= 0.5, 1, 0) 
slt2_predictions$h2oNormPreds  <- ifelse( slt2_predictions$RFH2O_norm_P >= 0.5, 1, 0)  
slt2_predictions$origVsH2O <- NA
slt2_predictions$origVsOrigScaled <- NA
slt2_predictions$origVsOrigNorm <- NA
slt2_predictions$origVsH2OScaled <- NA
slt2_predictions$origVsH2ONorm <- NA
for( i in 1:nrow(slt2_predictions) ){
  # Based on the way we are feeding the values to the compareAnswers function, 
  # Similiarities = OnlyA means that only the Orig RF got the right answer
  # Similiarities = OnlyB means that only the other RF got the right answer
  # All other answers will tell us where both models were right("BOTH_RIGHT"), or
  # wrong ("BOTH_WRONG")
  slt2_predictions[i,]$origVsOrigScaled <- compareAnswers(
    slt2_predictions[i,]$origPreds, slt2_predictions[i,]$origScaledPreds,slt2_predictions[i,]$Class )
  slt2_predictions[i,]$origVsOrigNorm <- compareAnswers(
    slt2_predictions[i,]$origPreds, slt2_predictions[i,]$origNormPreds, slt2_predictions[i,]$Class )
  slt2_predictions[i,]$origVsH2O <- compareAnswers(
    slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oPreds, slt2_predictions[i,]$Class )
  slt2_predictions[i,]$origVsH2OScaled <- compareAnswers(
    slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oScaledPreds, slt2_predictions[i,]$Class )
  slt2_predictions[i,]$origVsH2ONorm <- compareAnswers(
    slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oNormPreds, slt2_predictions[i,]$Class )
} 
rbind( head(slt2_predictions, 5), tail(slt2_predictions, 5) )

str(slt2_predictions)

'data.frame':   1986 obs. of  25 variables:
 $ SS               : num  -48.3 -8.8 -37.4 -26.3 -41.7 ...
 $ Pos10wrtsRNAStart: int  -56 -14 119 -15 -23 -48 -36 77 -15 -14 ...
 $ DistTerm         : int  0 1000 0 0 1000 1000 112 0 0 0 ...
 $ Distance         : int  0 -24 -98 -47 -2 -23 -53 -22 -71 -28 ...
 $ sameStrand       : int  1 0 0 1 0 0 0 0 1 0 ...
 $ DownDistance     : int  14 0 20 17 1 47 245 0 0 175 ...
 $ sameDownStrand   : int  1 0 0 1 1 0 0 0 1 0 ...
 $ Class            : num  1 1 1 1 1 1 1 1 1 1 ...
 $ OrigRF_P         : num  0.73 0.383 0.775 0.985 0.57 ...
 $ OrigRF_scaled_P  : num  0.745 0.362 0.72 0.98 0.58 ...
 $ OrigRF_norm_P    : num  0.743 0.328 0.748 0.98 0.61 ...
 $ RF_H2O_P         : num  0.74 0.372 0.705 0.973 0.539 ...
 $ RFH2O_scaled_P   : num  0.746 0.364 0.71 0.98 0.437 ...
 $ RFH2O_norm_P     : num  0.75 0.375 0.708 0.979 0.438 ...
 $ origPreds        : num  1 0 1 1 1 1 1 1 1 1 ...
 $ origScaledPreds  : num  1 0 1 1 1 1 1 1 1 1 ...
 $ origNormPreds    : num  1 0 1 1 1 1 1 1 1 1 ...
 $ h2oPreds         : num  1 0 1 1 1 1 1 1 1 1 ...
 $ h2oScaledPreds   : num  1 0 1 1 0 1 1 1 1 1 ...
 $ h2oNormPreds     : num  1 0 1 1 0 1 1 1 1 1 ...
 $ origVsH2O        : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsOrigScaled : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsOrigNorm   : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsH2OScaled  : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsH2ONorm    : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...

As an additional check, we also reviewed the correlations between the OrigRF predictions, and the alternative models’s predictions with the SLT2 predictions.

cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RF_H2O_P"])

[1] 0.9842173

cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"OrigRF_scaled_P"])

[1] 0.9979076

cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"OrigRF_norm_P"])

[1] 0.9976877

cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RFH2O_scaled_P"])

[1] 0.9846473

cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RFH2O_norm_P"])

[1] 0.9847233

10.2 Compare OrigRF vs the other Models with the SLT2 data

Here, we are simply taking a look at how many instances the OrigRF model got correct vs the other models, and how many instances both models predicted correctly or wrong. * A value of OnlyA means that only the orig RF got the right answer * A value of OnlyB means that only the other RF got the right answer * All other answers will tell us where both models were right (“BOTH_RIGHT”) or wrong (“BOTH_WRONG”)

10.2.2 Compare OrigRF vs OrigRF Scaled Model

nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "OnlyA",] )

[1] 8

nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "OnlyB",] )

[1] 8

nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_WRONG",] )

[1] 161

nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT",] )

[1] 1809

10.2.3 Compare OrigRF vs OrigRF Norm Model

nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "OnlyA",] )

[1] 8

nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "OnlyB",] )

[1] 12

nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_WRONG",] )

[1] 157

nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_RIGHT",] )

[1] 1809

10.2.4 Compare OrigRF vs H2O Model

nrow( slt2_predictions[slt2_predictions$origVsH2O == "OnlyA",] )

[1] 26

nrow( slt2_predictions[slt2_predictions$origVsH2O == "OnlyB",] )

[1] 19

nrow( slt2_predictions[slt2_predictions$origVsH2O == "BOTH_WRONG",] )

[1] 150

nrow( slt2_predictions[slt2_predictions$origVsH2O == "BOTH_RIGHT",] )

[1] 1791

10.2.5 Compare OrigRF vs H2O Model Scaled

nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "OnlyA",] )

[1] 29

nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "OnlyB",] )

[1] 19

nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_WRONG",] )

[1] 150

nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_RIGHT",] )

[1] 1788

10.2.6 Compare OrigRF vs H2O Model Normalized

nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "OnlyA",] )

[1] 32

nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "OnlyB",] )

[1] 19

nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_WRONG",] )

[1] 150

nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_RIGHT",] )

[1] 1785

11. LU Comparisons

11.1 Build LU Predictions table

lu_predictions <-  cbind(as.data.frame(ludata),               # Input
                           as.data.frame(origRF_lu_pred[,2]),   # Orig RF predictions 
                           as.data.frame(origRF_scaled_lu_pred[,2]),   # Orig RF Scaled predictions 
                           as.data.frame(origRF_norm_lu_pred[,2]),   # Orig RF Normalized predictions   
                           as.data.frame(rfh2o_lu_pred[,3]),        # H2O RF predictions
                           as.data.frame(rfh2o_scaled_lu_pred[,3]), # Scaled RF predictions
                           as.data.frame(rfh2o_norm_lu_pred[,3])    # Normalized RF predictions
)
colnames(lu_predictions) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance", 
                                "sameStrand", "DownDistance", "sameDownStrand", "Class",
                                "OrigRF_P","OrigRF_scaled_P","OrigRF_norm_P",
                                "RF_H2O_P","RFH2O_scaled_P","RFH2O_norm_P")
                                
lu_predictions$origPreds <- ifelse( lu_predictions$OrigRF_P >= 0.5, 1, 0) 
lu_predictions$origScaledPreds <- ifelse( lu_predictions$OrigRF_scaled_P >= 0.5, 1, 0) 
lu_predictions$origNormPreds <- ifelse( lu_predictions$OrigRF_norm_P >= 0.5, 1, 0) 
lu_predictions$h2oPreds  <- ifelse( lu_predictions$RF_H2O_P >= 0.5, 1, 0) 
lu_predictions$h2oScaledPreds  <- ifelse( lu_predictions$RFH2O_scaled_P >= 0.5, 1, 0) 
lu_predictions$h2oNormPreds  <- ifelse( lu_predictions$RFH2O_norm_P >= 0.5, 1, 0)  
lu_predictions$origVsH2O <- NA
lu_predictions$origVsOrigScaled <- NA
lu_predictions$origVsOrigNorm <- NA
lu_predictions$origVsH2OScaled <- NA
lu_predictions$origVsH2ONorm <- NA
for( i in 1:nrow(lu_predictions) ){
  # Based on the way we are feeding the values to the compareAnswers function, 
  # Similiarities = OnlyA means that only the Orig RF got the right answer
  # Similiarities = OnlyB means that only the other RF got the right answer
  # All other answers will tell us where both models were right("BOTH_RIGHT"), or
  # wrong ("BOTH_WRONG")
  lu_predictions[i,]$origVsOrigScaled <- compareAnswers(
    lu_predictions[i,]$origPreds, lu_predictions[i,]$origScaledPreds,lu_predictions[i,]$Class )
  lu_predictions[i,]$origVsOrigNorm <- compareAnswers(
    lu_predictions[i,]$origPreds, lu_predictions[i,]$origNormPreds,  lu_predictions[i,]$Class )
  lu_predictions[i,]$origVsH2O <- compareAnswers(
    lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oPreds,       lu_predictions[i,]$Class )
  lu_predictions[i,]$origVsH2OScaled <- compareAnswers(
    lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oScaledPreds, lu_predictions[i,]$Class )
  lu_predictions[i,]$origVsH2ONorm <- compareAnswers(
    lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oNormPreds,   lu_predictions[i,]$Class )
} 
str(lu_predictions)

'data.frame':   3009 obs. of  25 variables:
 $ SS               : num  -174.1 -30.9 -117.6 -30 -33.1 ...
 $ Pos10wrtsRNAStart: int  -18 -1000 -24 -26 56 -94 -35 12 77 244 ...
 $ DistTerm         : int  14 445 36 1000 1000 9 6 494 19 1000 ...
 $ Distance         : int  -14 -1217 -292 -121 -112 -32 -83 -65 0 -121 ...
 $ sameStrand       : int  1 0 0 0 0 1 1 1 1 1 ...
 $ DownDistance     : int  68 379 18 228 61 74 71 41 0 131 ...
 $ sameDownStrand   : int  0 1 1 1 1 1 1 1 1 0 ...
 $ Class            : num  1 1 1 1 1 1 1 1 1 1 ...
 $ OrigRF_P         : num  0.968 0.295 0.892 0.855 0.695 ...
 $ OrigRF_scaled_P  : num  0.948 0.305 0.897 0.895 0.72 ...
 $ OrigRF_norm_P    : num  0.958 0.285 0.91 0.83 0.738 ...
 $ RF_H2O_P         : num  0.888 0.286 0.879 0.869 0.648 ...
 $ RFH2O_scaled_P   : num  0.882 0.301 0.877 0.849 0.613 ...
 $ RFH2O_norm_P     : num  0.885 0.302 0.87 0.851 0.622 ...
 $ origPreds        : num  1 0 1 1 1 1 1 1 0 0 ...
 $ origScaledPreds  : num  1 0 1 1 1 1 1 1 0 0 ...
 $ origNormPreds    : num  1 0 1 1 1 1 1 1 0 0 ...
 $ h2oPreds         : num  1 0 1 1 1 1 1 1 0 0 ...
 $ h2oScaledPreds   : num  1 0 1 1 1 1 1 1 0 0 ...
 $ h2oNormPreds     : num  1 0 1 1 1 1 1 1 0 0 ...
 $ origVsH2O        : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsOrigScaled : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsOrigNorm   : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsH2OScaled  : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
 $ origVsH2ONorm    : chr  "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...

rbind( head(lu_predictions, 5), tail(lu_predictions, 5))

As an additional check, we also review the correlations between the OrigRF predictions, and the alternative models’ predictions with the LU data.

cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RF_H2O_P"])

[1] 0.9870177

cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"OrigRF_scaled_P"])

[1] 0.9977671

cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"OrigRF_norm_P"])

[1] 0.9980152

cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RFH2O_scaled_P"])

[1] 0.9863546

cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RFH2O_norm_P"])

[1] 0.9866413

11.2 Compare OrigRF vs the other Models with the LU data

Basically, the same explanation as in section 10.2 * A value of OnlyA means that only the orig RF got the right answer * A value of OnlyB means that only the other RF got the right answer * All other answers will tell us where both models were right (“BOTH_RIGHT”) or wrong (“BOTH_WRONG”)

11.2.1 Compare OrigRF vs OrigRF Scaled Model

nrow( lu_predictions[lu_predictions$origVsOrigScaled == "OnlyA",] )

[1] 9

nrow( lu_predictions[lu_predictions$origVsOrigScaled == "OnlyB",] )

[1] 19

nrow( lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_WRONG",] )

[1] 398

nrow( lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_RIGHT",] )

[1] 2583

11.2.2 Compare OrigRF vs OrigRF Norm Model

nrow( lu_predictions[lu_predictions$origVsOrigNorm == "OnlyA",] )

[1] 13

nrow( lu_predictions[lu_predictions$origVsOrigNorm == "OnlyB",] )

[1] 20

nrow( lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_WRONG",] )

[1] 397

nrow( lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_RIGHT",] )

[1] 2579

11.2.3 Compare OrigRF vs H2O Model

nrow( lu_predictions[lu_predictions$origVsH2O == "OnlyA",] )

[1] 35

nrow( lu_predictions[lu_predictions$origVsH2O == "OnlyB",] )

[1] 33

nrow( lu_predictions[lu_predictions$origVsH2O == "BOTH_WRONG",] )

[1] 384

nrow( lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT",] )

[1] 2557

11.2.4 Compare OrigRF vs H2O Model Scaled

nrow( lu_predictions[lu_predictions$origVsH2OScaled == "OnlyA",] )

[1] 31

nrow( lu_predictions[lu_predictions$origVsH2OScaled == "OnlyB",] )

[1] 35

nrow( lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_WRONG",] )

[1] 382

nrow( lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_RIGHT",] )

[1] 2561

11.2.5 Compare OrigRF vs H2O Model Normalized

nrow( lu_predictions[lu_predictions$origVsH2ONorm == "OnlyA",] )

[1] 29

nrow( lu_predictions[lu_predictions$origVsH2ONorm == "OnlyB",] )

[1] 35

nrow( lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_WRONG",] )

[1] 382

nrow( lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_RIGHT",] )

[1] 2563

IV) COMPARE MODEL’S METRICS

While sections 11.2 and 10.2 suggest that all the models are behaving similarly, making a judgement solely on accuracy can be deceiving. For this reason, we also decided to dive deeper into the model’s performances, and do additional side by side comparison of other performance metrics.

12. Get Performance Metrics

This evaluateData function was copied directly from the source materials(the additional material included in the original article), as it was the one used to evaluate the original model, and we’ve simply added comments a few extra comments. The H2O library already contains tools to obtain performance metrics from its models, so no additional functions were necessary with the H2O ML models.

evaluateData <- function(RF, data, labels){
  require(ROCR)
  require(PRROC)
  require(randomForest)
  res <- list()
  res$predD <- predict(RF, data, type = "prob")                        # Predictions as Generated by the model
  res$pred <- prediction(res$predD[,2], as.logical(labels))            # Predictions, but converted to S4 Object          
  res$PR <- performance(res$pred, measure = "prec", x.measure = "rec") # Precision Recall Curve
  res$SS <- performance(res$pred, measure="sens", x.measure="spec")    # Sensitivity vs. Specificity
  res$auc  <- performance(res$pred, measure = "auc")                   # Sensitivity vs. Specificity AUC
  res$acc <-  performance(res$pred, measure = "acc")                   # Accuracy
  res$pr <- pr.curve(scores.class0 = res$predD[labels == 1,2],         # Precision vs. Recall 
                     scores.class1 = res$predD[labels == 0,2], 
                     curve = T)
  return(res)
}
# OrigRF Performance Metrics
origRF_slt2_performance <- evaluateData(origRF, slt2data[,-8], slt2data[,8])
origRF_scaled_slt2_performance <- evaluateData(origRF_scaled, slt2data_scaled[,-8], slt2data_scaled[,8])
origRF_norm_slt2_performance <- evaluateData(origRF_norm, slt2data_norm[,-8], slt2data_norm[,8])
origRF_lu_performance <- evaluateData(origRF, ludata[,-8], ludata[,8])
origRF_scaled_lu_performance <- evaluateData(origRF_scaled, ludata_scaled[,-8], ludata_scaled[,8])
origRF_norm_lu_performance <- evaluateData(origRF_norm, ludata_norm[,-8], ludata_norm[,8])
# H2O RF Performance Metrics
# H2O provides a way to retrieve most of the desired metrics
# http://docs.h2o.ai/h2o/latest-stable/h2o-r/docs/reference/h2o.metric.html
slt2data_h2o <- slt2data
slt2data_h2o[,"Class"] <- as.logical(slt2data_h2o[,"Class"]) 
rfh2o_slt2_performance <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o))
ludata_h2o <- ludata
ludata_h2o[,"Class"] <- as.logical(ludata_h2o[,"Class"]) 
rfh2o_lu_performance <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o))
slt2data_h2o_scaled <- slt2data_scaled
slt2data_h2o_scaled[,"Class"] <- as.logical(slt2data_h2o_scaled[,"Class"]) 
rfh2o_slt2_performance_scaled <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o_scaled))
ludata_h2o_scaled <- ludata_scaled
ludata_h2o_scaled[,"Class"] <- as.logical(ludata_h2o_scaled[,"Class"]) 
rfh2o_lu_performance_scaled <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o_scaled))
slt2data_h2o_norm <- slt2data_norm
slt2data_h2o_norm[,"Class"] <- as.logical(slt2data_h2o_norm[,"Class"]) 
rfh2o_slt2_performance_norm <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o_norm))
ludata_h2o_norm <- ludata_norm
ludata_h2o_norm[,"Class"] <- as.logical(ludata_h2o_norm[,"Class"]) 
rfh2o_lu_performance_norm <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o_norm))

13. Compare Metrics

13.1 Accuracy

13.1.1 Accuracy Table

metrics_table <- data.frame("Accuracy" = 1:12)
rownames(metrics_table) <- c("origRF_slt2_perf", "origRF_scaled_slt2_perf", "origRF_norm_slt2_perf",
                             "rfh2o_slt2_perf", "rfh2o_slt2_perf_scaled", "rfh2o_slt2_perf_norm",
                             "origRF_lu_perf", "origRF_scaled_lu_perf", "origRF_norm_lu_perf", 
                             "rfh2o_lu_perf",  "rfh2o_lu_perf_scaled",  "rfh2o_lu_perf_norm")
metrics_table["origRF_slt2_perf","Accuracy"]        <- nrow(slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT" | slt2_predictions$origVsOrigScaled == "OnlyA",] )/
  nrow(slt2_predictions)
metrics_table["origRF_scaled_slt2_perf","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT" | slt2_predictions$origVsOrigScaled == "OnlyB",] )/
  nrow(slt2_predictions)
metrics_table["origRF_norm_slt2_perf","Accuracy"]   <- nrow(slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_RIGHT" | slt2_predictions$origVsOrigNorm == "OnlyB",] )/
  nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf","Accuracy"]         <- nrow(slt2_predictions[slt2_predictions$origVsH2O == "BOTH_RIGHT" | slt2_predictions$origVsH2O == "OnlyB",] )/
  nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf_scaled","Accuracy"]  <- nrow(slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_RIGHT" |  slt2_predictions$origVsH2OScaled == "OnlyB",])/
  nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf_norm","Accuracy"]    <- nrow(slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_RIGHT" | slt2_predictions$origVsH2ONorm == "OnlyB",])/
  nrow(slt2_predictions)
metrics_table["origRF_lu_perf","Accuracy"]          <- nrow(lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT" | lu_predictions$origVsH2O == "OnlyA",] )/
  nrow(lu_predictions)
metrics_table["origRF_scaled_lu_perf","Accuracy"]   <- nrow(lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_RIGHT" | lu_predictions$origVsOrigScaled == "OnlyB",] )/
  nrow(lu_predictions)
metrics_table["origRF_norm_lu_perf","Accuracy"]     <- nrow(lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_RIGHT" | lu_predictions$origVsOrigNorm == "OnlyB",] )/
  nrow(lu_predictions)
metrics_table["rfh2o_lu_perf","Accuracy"]           <- nrow(lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT" |  lu_predictions$origVsH2O == "OnlyB",] )/
  nrow(lu_predictions)
metrics_table["rfh2o_lu_perf_scaled","Accuracy"]    <- nrow(lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_RIGHT" | lu_predictions$origVsH2OScaled == "OnlyB",])/
  nrow(lu_predictions)
metrics_table["rfh2o_lu_perf_norm","Accuracy"]      <- nrow(lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_RIGHT" | lu_predictions$origVsH2ONorm == "OnlyB",] )/
  nrow(lu_predictions)
metrics_table

13.1.2 Accuracy graphs based on different thresholds(“cutoff”)

The Legend for the graphs: * Blue = OrigRF * Magenta = OrigRF but scaled * Black = OrigRF but normalized * Red = H2O RF model * Brown = H2O RF model with scaled data * Green = H2O RF model with normalized data

# SLT2 Accuracy Plot
plot(origRF_slt2_performance$acc, col = "blue", lwd = 5, main = "SLT2 Accuracy") 
lines(as.double(unlist(origRF_scaled_slt2_performance$acc@x.values)), as.double(unlist(origRF_scaled_slt2_performance$acc@y.values)), col = "magenta", lwd = 4)

lines(as.double(unlist(origRF_norm_slt2_performance$acc@x.values)), as.double(unlist(origRF_norm_slt2_performance$acc@y.values)), col = "black", lwd = 1) 
lines(h2o.accuracy(rfh2o_slt2_performance), type = "l", col = "red", lwd = 3)

lines(h2o.accuracy(rfh2o_slt2_performance_scaled), type = "l", col = "brown", lwd = 3)
lines(h2o.accuracy(rfh2o_slt2_performance_norm), type = "l", col = "green", lwd = 3)

# LU Accuracy Plot
plot(origRF_lu_performance$acc, col = "blue", lwd = 5, main = "LU Accuracy")
lines(as.double(unlist(origRF_scaled_lu_performance$acc@x.values)), as.double(unlist(origRF_scaled_lu_performance$acc@y.values)), col = "magenta", lwd = 4)

lines(as.double(unlist(origRF_norm_lu_performance$acc@x.values)), as.double(unlist(origRF_norm_lu_performance$acc@y.values)), col = "black", lwd = 1) 
lines(h2o.accuracy(rfh2o_lu_performance), type = "l", col = "red", lwd = 3)

lines(h2o.accuracy(rfh2o_lu_performance_scaled), type = "l", col = "brown", lwd = 3)
lines(h2o.accuracy(rfh2o_lu_performance_norm), type = "l", col = "green", lwd = 3)

13.2 AUCPR

metrics_table["origRF_slt2_perf","AUCPR"] <- origRF_slt2_performance$pr$auc.integral
metrics_table["origRF_scaled_slt2_perf","AUCPR"] <- origRF_scaled_slt2_performance$pr$auc.integral
metrics_table["origRF_norm_slt2_perf","AUCPR"] <- origRF_norm_slt2_performance$pr$auc.integral
metrics_table["rfh2o_slt2_perf", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance)
metrics_table["rfh2o_slt2_perf_scaled", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance_scaled)
metrics_table["rfh2o_slt2_perf_norm", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance_norm)
metrics_table["origRF_lu_perf",  "AUCPR"] <- origRF_lu_performance$pr$auc.integral
metrics_table["origRF_scaled_lu_perf",  "AUCPR"] <- origRF_scaled_lu_performance$pr$auc.integral
metrics_table["origRF_norm_lu_perf",  "AUCPR"] <- origRF_norm_lu_performance$pr$auc.integral
metrics_table["rfh2o_lu_perf",  "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance)
metrics_table["rfh2o_lu_perf_scaled", "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance_scaled)
metrics_table["rfh2o_lu_perf_norm", "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance_norm)
metrics_table

plot(origRF_slt2_performance$PR,  col = "blue", lwd = 5, main = "SLT2 PR Curve" )
lines(as.double(unlist(origRF_scaled_slt2_performance$PR@x.values)), as.double(unlist(origRF_scaled_slt2_performance$PR@y.values)),  col = "magenta", lwd = 4 )

lines(as.double(unlist(origRF_norm_slt2_performance$PR@x.values)), as.double(unlist(origRF_norm_slt2_performance$PR@y.values)),  col = "black", lwd = 2 )
lines(x = h2o.recall(rfh2o_slt2_performance)[,"tpr"],
      y = h2o.precision(rfh2o_slt2_performance)[,"precision"],
      col = "red", type = "l", lwd = 3
)

lines(x = h2o.recall(rfh2o_slt2_performance_scaled)[,"tpr"],
      y = h2o.precision(rfh2o_slt2_performance_scaled)[,"precision"],
      col = "brown", type = "l", lwd = 3
)
lines(x = h2o.recall(rfh2o_slt2_performance_norm)[,"tpr"],
      y = h2o.precision(rfh2o_slt2_performance_norm)[,"precision"],
      col = "green", type = "l", lwd = 3
)

plot(origRF_lu_performance$PR,  col = "blue", lwd = 5, main = "LU PR Curve" )
lines(as.double(unlist(origRF_scaled_lu_performance$PR@x.values)), as.double(unlist(origRF_scaled_lu_performance$PR@y.values)),  col = "magenta", lwd = 4 )

lines(as.double(unlist(origRF_norm_lu_performance$PR@x.values)), as.double(unlist(origRF_norm_lu_performance$PR@y.values)),  col = "black", lwd = 2 )
lines(x = h2o.recall(rfh2o_lu_performance)[,"tpr"],
      y = h2o.precision(rfh2o_lu_performance)[,"precision"],
      col = "red", type = "l", lwd = 3
)

lines(x = h2o.recall(rfh2o_lu_performance_scaled)[,"tpr"],
      y = h2o.precision(rfh2o_lu_performance_scaled)[,"precision"],
      col = "brown", type = "l", lwd = 3
)
lines(x = h2o.recall(rfh2o_lu_performance_norm)[,"tpr"],
      y = h2o.precision(rfh2o_lu_performance_norm)[,"precision"],
      col = "green", type = "l", lwd = 3
)

13.3 Sensitivy vs Specificity Graph

plot(origRF_slt2_performance$SS, col = "blue", lwd = 5, main = "SLT2 Sensitivity vs Specificity")
lines(as.double(unlist(origRF_scaled_slt2_performance$SS@x.values)), as.double(unlist(origRF_scaled_slt2_performance$SS@y.values)),  col = "magenta", lwd = 4 )

lines(as.double(unlist(origRF_norm_slt2_performance$SS@x.values)), as.double(unlist(origRF_norm_slt2_performance$SS@y.values)),  col = "black", lwd = 2 )
lines(x = h2o.specificity(rfh2o_slt2_performance)[,"tnr"],
     y = h2o.sensitivity(rfh2o_slt2_performance)[,"tpr"],
     col = "red", type = "l", lwd = 3
     )

lines(x = h2o.specificity(rfh2o_slt2_performance_scaled)[,"tnr"],
      y = h2o.sensitivity(rfh2o_slt2_performance_scaled)[,"tpr"],
      col = "brown", type = "l", lwd = 3
)
lines(x = h2o.specificity(rfh2o_slt2_performance_norm)[,"tnr"],
      y = h2o.sensitivity(rfh2o_slt2_performance_norm)[,"tpr"],
      col = "green", type = "l", lwd = 3
)

plot(origRF_lu_performance$SS, col = "blue", lwd = 5, main = "LU Sensitivity vs Specificity")
lines(as.double(unlist(origRF_scaled_lu_performance$SS@x.values)), as.double(unlist(origRF_scaled_lu_performance$SS@y.values)),  col = "magenta", lwd = 4 )

lines(as.double(unlist(origRF_norm_lu_performance$SS@x.values)), as.double(unlist(origRF_norm_lu_performance$SS@y.values)),  col = "black", lwd = 2 )
lines(x = h2o.specificity(rfh2o_lu_performance)[,"tnr"],
      y = h2o.sensitivity(rfh2o_lu_performance)[,"tpr"],
      col = "red", type = "l", lwd = 3
)

lines(x = h2o.specificity(rfh2o_lu_performance_scaled)[,"tnr"],
      y = h2o.sensitivity(rfh2o_lu_performance_scaled)[,"tpr"],
      col = "brown", type = "l", lwd = 3
)
lines(x = h2o.specificity(rfh2o_lu_performance_norm)[,"tnr"],
      y = h2o.sensitivity(rfh2o_lu_performance_norm)[,"tpr"],
      col = "green", type = "l", lwd = 3
)

13.4 Other H2O Metrics

Some additional information that’s easily obtainable from the H2O library.

h2o.confusionMatrix(rfh2o_slt2_performance)

Confusion Matrix (vertical: actual; across: predicted)  for max f1 @ threshold = 0.644166666567326:

h2o.confusionMatrix(rfh2o_slt2_performance_scaled)

Confusion Matrix (vertical: actual; across: predicted)  for max f1 @ threshold = 0.232701931446791:

h2o.confusionMatrix(rfh2o_slt2_performance_norm)

Confusion Matrix (vertical: actual; across: predicted)  for max f1 @ threshold = 0.266167617663741:

plot(h2o.F1(rfh2o_slt2_performance), col = "blue", main = "F1 Performance")
lines(h2o.F1(rfh2o_slt2_performance_scaled), col = "red")

lines(h2o.F1(rfh2o_slt2_performance_norm), col = "orange")

plot(rfh2o_slt2_performance, # REMINDER: TPR = Sensitivity, FPR = (1 - specificity)
     type = "roc", 
     col = "red",
     cex = 0.2,
     pch = 10
)

h2o.confusionMatrix(rfh2o_lu_performance)

Confusion Matrix (vertical: actual; across: predicted)  for max f1 @ threshold = 0.496151700350456:

plot(h2o.F1(rfh2o_lu_performance))

plot(rfh2o_lu_performance,
     type = "roc", 
     col = "red",
     cex = 0.2,
     pch = 10
)

V) Random Forest Explainer (PDP)

Random Forest Explainer is a global interpretability model that ultimately allows you to better understand your RF model and create visually attractive and useful PDP plots. The methodology for behind RFE is simple: identify the top most important features, their interactions, and then graph these interactions in what is essentially a 2D PDP plot. Unfortunately, when trying to use RFE within the notebook, the outputs generated errors and the plots were simply not showing. To get around this problem, we will simply add the graphs as generated by the RFE library. The code that generated these graphs be found in here, and the output html generated by the explain_forest() function can also be found “./Your_forest_explained.html”. Keep in mind that the HTML file is automatically generated by the library and does not include all the plots we generated (it seems to only show what it considers the main plots). Since the you still have to generate the plots yourself, the HTML file is a cool bonus, but not a necessary.

14. Variable Importance Measures

14.1 Distribution of Minimal Depth and its Mean

Distribution of minimal depth and its mean

14.2 Importance Measures Table

rfe_importance_frame <- read.csv("./Results/RFE_importance_frame_origRF.csv", sep = ",", header = TRUE)
rfe_importance_frame

14.3 Multiway Importance Plots

14.3.1 Times a root vs Mean min depth vs Number of Nodes

Times a root vs Mean min depth vs Number of Nodes

14.3.2 Accuracy Decrease vs. Gini Decrease vs. Times a Root

Accuracy Decrease vs. Gini Decrease vs. Times a Root ## 15. GGpairs Comparisons The following graphs are simply helpful in identifying feature importance

ggpairs full

ggparis main features

ggpairs importance rankings

16. Feature Interactions

16.1 Top 30 Feature Interactions

Top 30 Feature Interactions

From this graph, we gather that the top 5 feature interactions are: 1. DownDistance:SS 1. Distance:SS 1. DownDistance:Pos10wrtsRNAStart, 1. DistTerm:SS, 1. Distance:Pos10wrtsRNAStart Even though Distance and DownDistance were considered the most important features, their interactions did not appear to be as significant as the others

16.2 DownDistance vs. SS

DownDistance vs. SS * The higher(closer to 0) the SS, the higher the confidence in the prediction

16.3 Distance vs. SS

Distance vs. SS * distances greater than -850 => higher chances of having a bona fide sRNA * SS higher than -10 => Lower the prob of having a bona fide sRNA

16.4 DownDistance vs. Pos10wrtsRNAStart

DownDistance vs. Pos10wrtsRNAStart * No clear disernable takeaway

16.5 DistTerm vs. SS

DistTerm vs. SS * SS of 0 seems to virtually guarantee that we don’t have a bona fide sRNA

16.6 Distance vs. Pos10wrtsRNAStart

* Having a Pos10wrtsRNAStart near 0 or slightly lower seems to make predictions fuzzy (near 0.5)

16.7 Distance vs. DownDistance

Distance vs. DownDistance * These are the 2 top features, but their interactions are not among the top ones * When both features are close to 0, the chances of having a bona fide sRNA greatly increase dev.off()

16.8 DownDistance vs. DistTerm

Based on the previous plots, generating the following plots seem like a smart thing to do DownDistance vs. DistTerm * having a DistTerm less than 50, or close to 0, increases the chances of having a bona fide sRNA

16.9 Distance vs. DistTerm

Distance vs. DistTerm

having a Distance close to 0 increases the chances of having a bona fide sRNA

16.10 sameStrand vs sameDownStrand

sameStrand vs sameDownStrand

sameStrand and sameDownStrand were considered the least relevant/important features. However, the overall tone of the graph suggests that having these variables helps boosting the model’s confidence in any given prediction. (ie. compare the tones of the previous graph to this one, and it’s the model has more “confidence” with its overall decisions in this graph.)

17. RFE Conclusions

There are many conclusions that may be gathered or obtained from the plots above, but the main takeaways can be summarized as follows: * The closer the SS is to 0, the higher the model’s confidence in its decision * Having a Distance between -850 and 0 greatly increases the chances of having an sRNA * An SS of -12 or higher will almost guarantee a non bona fide sRNA * There is a sweet spot for finding bona fide sRNAs if the DownDistance is less than 500 and the Distance is less than 800 * Having a DownDistance less than 60 also greatly increases the chances of having an sRNA * While sameStrand and sameDownStrand were considered the least important features, they seem to contribute in increasing the model’s confidence (at least in classifying something as not being an sRNA, as the graph seems to contain mostly solid blue colors)

NOTE/DISCLAIMER: The numbers here are human approximations based solely on looking at the graphs generated by RFE, and RFE uses only the training data to generate its graphs. In other words, while the goal of the RF is to have a general model applicable to any possible bacterial sRNA, there is no guarantee that these conclusions apply to every species of bacteria in the world.

VI) LIME

LIME FUNCTION ARGUMENTS

x The training data used for training the model that should be explained.
model The model whose output should be explained
bin_continuous Should continuous variables be binned when making the explanation
n_bins The number of bins for continuous variables if bin_continuous = TRUE
quantile_bins Should the bins be based on n_bins quantiles or spread evenly over the range of the training data
use_density If bin_continuous = FALSE should continuous data be sampled using a kernel density estimation. If not, continuous features are expected to follow a normal distribution.

LIME SETTINGS JUSTIFICATIONS

NOTE: many of these settings were obtained through trial and error, and/or based on the suggestions provided by LIME and its libraries. Since LIME’s results are not always the same, we used the settings that generated the most consistent and valid explanations during our experinments. Of course, we encourage others to try out other settings and experinment further. * Settings for “explainer” * bin_continuous = TRUE –> Makes explanation easier, and having this setting as true helped in obtaining more consistent and reliable results. * quantile_bins = FALSE –> setting this to TRUE generates an error b/c sameStrand and sameDownStrand don’t have enough variance for the algorithm to use quantile bining.
* use_density = TRUE –> with bin_continuous set to TRUE, this setting becomes irrelevant. In section 3.4, we showed that the features don’t follow a normal distribution. We tried setting bin_continuous = FALSE and use_density = TRUE, but this generated contradictory explanations that didn’t make sense. * n_bins = 10 –> The default value wasn’t giving us consistent results, so we bumped up the number to 10 and started getting seemingly good results

Settings for “explain”
- n_labels = 1 –> we only have 1 label
- n_features = 7 –> we wanted our explanations to be based on all 7 features
- n_permutations = 2000 –> This number was set through trial and error. Having a small number of permutations does not generate consisten results, so a high number of permutations is recommended. 2000 just seemed to have yielded good results, so we decided to keep this value. If speed is a concern, lowering this number to 1000 should still yield good results.
- dist_fun = “gower” –> LIME’s default dist_fun, and since we have categorical features (sameStrand and sameDownStrand), according to LIME’s documentation, “gower” is the functions of choice when dealing with a mixture of numerical and categorical features
- Side Note: We tried a couple of runs of the script with euclidean distance, but it didn’t seem to have much of an impact. Additionally, the different types of settings, such as different gower powers, kernel widths, and distance functions, that can be tweaked in LIME means that a 100% thorough analysis of every possibility may be nearly impossible. For this reason, we tried to stick to the default settings as offered by LIME.

18. LIME PRE-SETUP

18.1 LIME compatability function

The original model was built using the randomForest library found in CRAN. However, even though LIME is supposed to be model agnostic, it’s current R implementation only supports certain models out of the box. To extend support of the LIME library to other models, the following functions need to be defined for each unsupported model type.

model_type.randomForest <- function(x,...) 'classification'
predict_model.randomForest <- function(x, newdata, type, ...) {
  res <- predict(x, newdata = newdata, type = "prob")
  switch(
    type,
    raw = data.frame(Response = ifelse(as.vector(res[,2]) > 0.5, "1", "0"), stringsAsFactors = FALSE),
    prob = as.data.frame(res, check.names = F) 
  )
}

18.2 Data to Explain

sampleData, sampleData_scaled, and sampleData_norm all contain the same features, and hence, we should be getting similar explanations in all of our results. We also apply LIME to the same instance 4 times, with the purpose of testing the validity and consistency of LIME’s explanations.

sampleData <- slt2data[1,]  # A true sample
sampleData[c(2:4),] <- slt2data[1,]
sampleData[5,] <- slt2data[1986,] # A false sample
sampleData[c(6:8),] <- slt2data[1986,]
sampleData

sampleData_scaled <- slt2data_scaled[1,]  
sampleData_scaled[c(2:4),] <- slt2data_scaled[1,]
sampleData_scaled[5,] <- slt2data_scaled[1986,] 
sampleData_scaled[c(6:8),] <- slt2data_scaled[1986,]
sampleData_scaled

sampleData_norm <- slt2data_norm[1,]  
sampleData_norm[c(2:4),] <- slt2data_norm[1,]
sampleData_norm[5,] <- slt2data_norm[1986,] 
sampleData_norm[c(6:8),] <- slt2data_norm[1986,]
sampleData_norm

19. LIME with RandomForest Library

19.1 In original Model as is

While on occassion all results may look similar, we recommend running this particular section a few times, and you will notice significant changes, like features that may support the explanation in one plot, but contradict it in the next.

predict(origRF, sampleData[c(1,5),-c(8:9)], type = "prob")

                             0     1
Thr_leader               0.270 0.730
S_enterica_LT2_RAND_1598 0.965 0.035
attr(,"class")
[1] "matrix" "votes"

lime_explainer_orig <- lime(as.data.frame(trainData[,c(1:7)]), origRF,
                            bin_continuous = TRUE, quantile_bins=FALSE, use_density = TRUE)
lime_explanations_orig <- explain(as.data.frame(sampleData[c(1:4),c(1:7)]), lime_explainer_orig, n_labels = 1, n_features = 7, n_permutations = 2000)
plot_features(lime_explanations_orig)

lime_explanations_orig <- explain(as.data.frame(sampleData[c(5:8),c(1:7)]), lime_explainer_orig, n_labels = 1, n_features = 7, n_permutations = 2000)

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_orig)

19.2 In original scaled Model

predict(origRF_scaled, sampleData_scaled[c(1,5),-c(8:9)], type = "prob")

                         FALSE  TRUE
Thr_leader               0.255 0.745
S_enterica_LT2_RAND_1598 0.950 0.050
attr(,"class")
[1] "matrix" "votes"

lime_explainer_orig_scaled <- lime(as.data.frame(trainData_scaled[, c(1:7)]), origRF_scaled,
                                   bin_continuous = TRUE, quantile_bins=FALSE, 
                                   use_density = FALSE, n_bins = 10)
norm_dist <- as.data.frame(lime_explainer_orig_scaled$bin_cuts)
lime_explainer_orig_scaled <- lime(as.data.frame(trainData_scaled[, c(1:7)]), origRF_scaled,
                                   bin_continuous = TRUE, quantile_bins=FALSE, 
                                   use_density = TRUE, n_bins = 10)
not_norm_dist <- as.data.frame(lime_explainer_orig_scaled$bin_cuts)
# The correlations here show that the bins used by the explainer are the same regardless of what value is given to the use_density parameter.
cor(norm_dist[,"SS"],not_norm_dist[,"SS"])

[1] 1

cor(norm_dist[,"Pos10wrtsRNAStart"],not_norm_dist[,"Pos10wrtsRNAStart"])

[1] 1

cor(norm_dist[,"DistTerm"],not_norm_dist[,"DistTerm"])

[1] 1

cor(norm_dist[,"Distance"],not_norm_dist[,"Distance"])

[1] 1

cor(norm_dist[,"sameStrand"],not_norm_dist[,"sameStrand"])

[1] 1

cor(norm_dist[,"DownDistance"],not_norm_dist[,"DownDistance"])

[1] 1

cor(norm_dist[,"sameDownStrand"],not_norm_dist[,"sameDownStrand"])

[1] 1

lime_explainer_orig_scaled$bin_cuts # All the features seem to be evenly split into 10, ie, a density is not used for the bin splitting.

$SS
 [1] -4.3406919 -3.7652111 -3.1897304 -2.6142496 -2.0387689 -1.4632881 -0.8878074 -0.3123266  0.2631541  0.8386349  1.4141156

$Pos10wrtsRNAStart
 [1] -1.9301214 -1.6047743 -1.2794272 -0.9540801 -0.6287330 -0.3033859  0.0219612  0.3473083  0.6726554  0.9980025  1.3233496

$DistTerm
 [1] -1.7364372 -1.4902626 -1.2440880 -0.9979134 -0.7517387 -0.5055641 -0.2593895 -0.0132149  0.2329597  0.4791343  0.7253090

$Distance
 [1] -14.828223 -13.324592 -11.820961 -10.317330  -8.813699  -7.310068  -5.806437  -4.302807  -2.799176  -1.295545   0.208086

$sameStrand
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$DownDistance
 [1] -0.1934291  1.7162357  3.6259006  5.5355654  7.4452302  9.3548950 11.2645599 13.1742247 15.0838895 16.9935543 18.9032192

$sameDownStrand
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

lime_explanations_orig_scaled <- explain(as.data.frame(sampleData_scaled[c(1:4),c(1:7)]), 
                                         lime_explainer_orig_scaled, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000
                                         )
plot_features(lime_explanations_orig_scaled)

lime_explanations_orig_scaled <- explain(as.data.frame(sampleData_scaled[c(5:8),c(1:7)]), 
                                         lime_explainer_orig_scaled, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000
                                         )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_orig_scaled)

19.3 In original normalized Model

predict(origRF_norm, sampleData_norm[c(1,5),-c(8:9)], type = "prob")

                          FALSE   TRUE
Thr_leader               0.2575 0.7425
S_enterica_LT2_RAND_1598 0.9500 0.0500
attr(,"class")
[1] "matrix" "votes"

lime_explainer_orig_norm <- lime(as.data.frame(trainData_norm[, c(1:7)]), origRF_norm,
                                   bin_continuous = TRUE, quantile_bins=FALSE, 
                                   use_density = TRUE, n_bins = 10)
lime_explainer_orig_norm$bin_cuts # all bins are split evenly, from 0 to 1, in increments of 0.1, as would be expected

$SS
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$Pos10wrtsRNAStart
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$DistTerm
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$Distance
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$sameStrand
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$DownDistance
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

$sameDownStrand
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

lime_explanations_orig_norm <- explain(as.data.frame(sampleData_norm[c(1:4),c(1:7)]), 
                                         lime_explainer_orig_norm, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000
                                         )
plot_features(lime_explanations_orig_norm)

lime_explanations_orig_norm <- explain(as.data.frame(sampleData_norm[c(5:8),c(1:7)]), 
                                         lime_explainer_orig_norm, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000
                                         )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_orig_norm)

20. LIME in Orig RF Conclusions

We tried applying LIME to the Orig RF(in part 15) hoping it would work like a plug-and-play application. To our surprise, you can’t get LIME to properly work with the “Original” RF model because you must normalize, or scale the data. LIME generates random permutations to the features in order to create a linear model, but if the features have different scales and values, it is plausible that features with higher scales will simply outweight the other features, even if they’re significantly less important, and hence create skewed models and explanations. To test this hypothesis, we created a GLM model and reviewed its R^2 value, which was suprisingly low, and hence, finished convincing us that the linear models generated by LIME with the OrigRF were inadequate. Additionally, since LIME randomizes the permutation points, based on our experinments, it seems that LIME is using its own randomization algorith (ie. we couldn’t set a seed for it), and hence, the exact results we obtained with the OrigRF may not be completely repeatable. However, for the particular case in part 15, that may be irrelevant as the point we are trying to prove there is that LIME’s explanations when the data is not scaled or normalized are inconsistent, and hence cannot be trusted.
For each LIME explanation plot, since we’re having LIME explain the same instance, all 4 plots should be identical to each other. However, with the OrigRF model, there’s usually at least 1 plot that ranks the feature contributions differently, or that may have a feature “Supporting” in one plot, but “Contradicting” in the next. Due to LIME’s nature, obtaining slightly different results was expected, but not to a degree that would make the explanations invalid.

By scaling, or normalizing, the data, we were able to obtain more consistent results (ie. features that support or contradict a prediction, behave the same in all the corresponding 4 plots, although their importance ranking may be different), and in most cases, the rankings of feature contributions as plotted by LIME actually remained the same. LIME behaves properly if the data is scaled and normalized.

An additional interesting take-away, is that sameStrand and sameDownStrand are considered the least important features, and seem to become completely irrelevant for predictions where the instance is FALSE as LIME simply exclude this feature .

21. LIME for the H2O RF models

Initial struggles with getting LIME set up with the original random forest lead us to believe that since RandomForest wasn’t a supported method by default in the Lime Library, that maybe using a supported method would help us overcome these problems. During our investigation, we discovered that H2O offered additional IMs (PDPs and SHAP), and that their algorithms were natively supported by the Lime library. Thinking we could kill 3 birds (test 3 IMs) with one stone (one package), we proceeded to create the equivalent proxy models in H2O and run the corresponding test. The first H2O RF model was created prior to realizing we needed to have normalized and scaled data sets, as these data preprocessing techniques are normally not necessary for random forests. Further investigations and testing allowed us to discover that normalizing and scaling the data allowed LIME to generate reasonable explanations with the OrigRF model. Eventually we decided to create 5 models in total, 2 with the original RandomForest library, and 3 with the H2O library. Even though the performance metrics for the scaled and normalized H2O RF models did not match those of the OrigRF, we decided to still include them in the following sections.

21.1 H2O RF Model

h2o.predict(object = rfh2o, newdata = as.h2o(sampleData[c(1,8),c(1:7)]) )


[2 rows x 3 columns]

lime_explainer_rfh2o <- lime( as.data.frame( trainData[,c(1:7)] ), rfh2o,
                              bin_continuous = TRUE, quantile_bins = FALSE,
                              n_bins = 10
)
lime_explanations_rfh2o <- explain( as.data.frame(sampleData[c(1:4),c(1:7)] ),lime_explainer_rfh2o, 
                                    n_labels = 1,  n_features = 7, 
                                    n_permutations = 2000 )
plot_features(lime_explanations_rfh2o)

lime_explanations_rfh2o <- explain( as.data.frame(sampleData[c(5:8),c(1:7)] ),lime_explainer_rfh2o, 
                                    n_labels = 1,  n_features = 7, 
                                    n_permutations = 2000 )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_rfh2o)

21.2 In H2O RF scaled Model

h2o.predict(object = rfh2o_scaled, newdata = as.h2o(sampleData_scaled[c(1,8),c(1:7)]))


[2 rows x 3 columns]

lime_explainer_rfh2o_scaled <- lime( as.data.frame( trainData_scaled[,c(1:7)] ), rfh2o_scaled,
                              bin_continuous = TRUE, quantile_bins = FALSE) 
lime_explanations_rfh2o_scaled <- explain( as.data.frame(sampleData_scaled[c(1:4),c(1:7)]),  
                                    lime_explainer_rfh2o_scaled, n_labels = 1, 
                                    n_features = 7, n_permutations = 2000, dist_fun = "gower" )
plot_features(lime_explanations_rfh2o_scaled)

lime_explanations_rfh2o_scaled <- explain( as.data.frame(sampleData_scaled[c(5:8),c(1:7)]),  
                                    lime_explainer_rfh2o_scaled, n_labels = 1, 
                                    n_features = 7, n_permutations = 2000, dist_fun = "gower" )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_rfh2o_scaled)

21.3 LIME to H2O RF normalized Model

h2o.predict(object = rfh2o_norm, newdata = as.h2o(sampleData_norm[c(1,8),c(1:7)]) )


[2 rows x 3 columns]

lime_explainer_rfh2o_norm <- lime( as.data.frame( trainData_norm[,c(1:7)] ), rfh2o_norm,
                                   bin_continuous = TRUE, quantile_bins = FALSE, use_density = TRUE)
lime_explanations_rfh2o_norm <- explain( as.data.frame(sampleData_norm[c(1:4),c(1:7)] ),
                                         lime_explainer_rfh2o_norm, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000, dist_fun = "gower" )
plot_features(lime_explanations_rfh2o_norm)

lime_explanations_rfh2o_norm <- explain( as.data.frame(sampleData_norm[c(5:8),c(1:7)] ),
                                         lime_explainer_rfh2o_norm, n_labels = 1, 
                                         n_features = 7, n_permutations = 2000, dist_fun = "gower" )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_rfh2o_norm)

21.4 Applying LIME to GLM Model (BONUS)

lime_explainer_glmh2o <- lime( as.data.frame( trainData[,c(1:7)] ),glmh2o, 
                               bin_continuous = TRUE, quantile_bins = FALSE, 
                               use_density = FALSE, n_bins = 10 )
lime_explanations_glmh2o <- explain( as.data.frame(sampleData[c(1:4),c(1:7)] ), lime_explainer_glmh2o,   
                                         n_labels = 1,  n_features = 7, 
                                         n_permutations = 2000, dist_fun = "gower"  )
plot_features(lime_explanations_glmh2o)

lime_explanations_glmh2o <- explain( as.data.frame(sampleData[c(5:8),c(1:7)] ), lime_explainer_glmh2o,   
                                         n_labels = 1,  n_features = 7, 
                                         n_permutations = 2000, dist_fun = "gower"  )

skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite range

plot_features(lime_explanations_glmh2o)

25. LIME CONCLUSIONS

A known problem with LIME is that explanations may be inconsistent from one instance to the next, even when the instances are very similar to each. With this in mind, each instance we tested was tested at least 4 times (we manually ran this section over and over again) , and with a high enough number of permutations, we managed to get somewhat consistent results. However, some of the results obtained from LIME were heavily flawed as sometimes it would yield explanations that contradicted the result. For some reason, when explaining false instances, LIME would tend to leave out sameStrand and sameDownStrand from the explanation. Additionally, a GLM was also trained to attempt to measure the variance of the data, but the results from the GLM yielded an R^2 value between 0.2 and 0.3, with a max recall of 0.07. In other words, the GLM model was only predicting false instances, and since LIME uses linear models for its explanations, based on our observations, it is safe to conclude that LIME was doing the same. Whenever a TRUE instance was analyze, LIME’s explanations would say that most factors contradict the prediction, which is obviously counter intuitive and useless.

During the making of this project, several problems were encountered while trying to apply LIME to the sRNA RF model. LIME attempts to explain a complex model’s behavior in a small region corresponding to a particular instance of interest by applying a linear model. However, in this particular use case, the mixture of categorical and numerical values, plus the differences in ranges within each feature resulted in various experinments without consistent results. For this particular use case, LIME was implement in models trained with the original training data, scaled training data, and normalized training data. Several attempts were also made at modifying the arguments for the lime() function, such as all the possible permutations between bin_continuous, quantile_bins and use_density. In the explain function, the number of permutations was increased from 10 to 2000 and the Euclidean and Gower distance functions were tested with kernel_widths ranging from 0.001 to 5 because we were not getting consistent

F) SHAP Values

SHAP_H2O1 <- h2o.predict_contributions(rfh2o, as.h2o(sampleData[,c(1:7)]))
head(SHAP_H2O1,10)

G) PDP

trainData[,c(1:9)]


[652 rows x 9 columns]

LS0tCnRpdGxlOiAiSW50ZXJwcmV0YWJpbGl0eSBNb2RlbHMgaW4gdGhlIFJhbmRvbSBGb3Jlc3Qgc1JOQSIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQKICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQKLS0tCiMgVGFibGUgb2YgQ29udGVudHMKMS4gUFJFLVNFVFVQID0gV2UgYXJlIHNpbXBseSBnZXR0aW5nIHRoZSBlbnZpcm9ubWVudCBhbmQgZGF0YSByZWFkeQoxLiBUUkFJTiBORVcgSDJPIE1PREVMID0gVG8gbWFrZSB1c2Ugb2YgSDJPJ3MgbGlicmFyaWVzLCB3ZSBuZWVkIHRvIGNyZWF0ZSB0aGUgbW9kZWxzIGluIEgyTyAoaWUuIG91ciBvcmlnaW5hbCBSRiBtb2RlbCBjYW5ub3QgYmUgZGlyZWN0bHkgY29udmVydGVkIHRvIGFuIGgybyBtb2RlbCkKMS4gQ09NUEFSRSBNT0RFTCdTIFBSRURJQ1RJT05TID0gV2UgaGF2ZSB0aGUgT3JpZ1JGIGFuZCBhbGwgdGhlIG90aGVyIG5ldyBtb2RlbHMgZ2VuZXJhdGUgcHJlZGljdGlvbnMgb24gdGhlIHRlc3RpbmcgZGF0YQoxLiBDT01QQVJFIE1PREVMJ1MgTUVUUklDUyA9IEEgdHJ1ZSB2YWxpZGF0aW9uIG9mIHRoZSBtb2RlbHMnIHBlcmZvcm1hbmNlLCBhbmQgYSBjb3JyZWN0IHdheSBvZiB2YWxpZGF0aW5nIGlmIG91ciBuZXcgbW9kZWxzIG1ha2UgZ29vZCBwcm94aWVzIGZvciB0aGUgb3JpZ2luYWwgbW9kZWwKMS4gTElNRSA9IEhlcmUgd2UgYWN0dWFsbHkgYmVnaW4gdG8gYXBwbHkgTElNRSB0byBvdXIgUkYgbW9kZWxzCjEuIFNIQVAgVmFsdWVzID0gV2UgdXNlIHRoZSBIMk8gbGlicmFyeSB0byBnZW5lcmF0ZSB0aGUgU0hBUCB2YWx1ZXMgZm9yIG91ciBIMk8gUkYgbW9kZWwKMS4gUERQcyA9IFNpbWlsYXIgdG8gdGhlIHByZXZpb3VzIHNlY3Rpb24sIHdlIHVzZSB0aGUgSDJPIGxpYnJhcnkgdG8gZ2VuZXJhdGUgdGhlIFBEUHMgZm9yIHRoZSBIMk8gUkYgbW9kZWwKCiMjIEludHJvZHVjdGlvbgpUaGUgZ29hbCBvZiB0aGlzIHByb2plY3Qgd2FzIHRvIGFwcGx5IGludGVycHJldGFiaWxpdHkgbW9kZWxzIHRvIHRoZSByYW5kb20gZm9yZXN0IGNsYXNzaWZpZXIgaW4gdGhlIFBlZXJKIGFydGljbGUgWyJQcmlvcml0aXppbmcgYm9uYSBmaWRlIGJhY3RlcmlhbCBzbWFsbCBSTkFzIHdpdGggbWFjaGluZSBsZWFybmluZyBjbGFzc2lmaWVycyJdKGh0dHBzOi8vcGVlcmouY29tL2FydGljbGVzLzYzMDQvKSB3aXRoIHRoZSBwdXJwb3NlcyBvZiBnYWluaW5nIGRlZXBlciBpbnNpZ2h0cyBpbnRvIHRoZSBwcmlvcml0aXphdGlvbiBvZiBib25hIGZpZGUgYmFjdGVyaWFsIHNSTkFzLiAKCgojIyBTb21lIGtleSBUZXJtcwoqIE9yaWdSRi9PcmlnIFJGID0gT3JpZ2luYWwgcmFuZG9tIGZvcmVzdCBhcyBmb3VuZCBpbiBbbGlua10oaHR0cHM6Ly9naXRodWIuY29tL0Jpb2luZm9ybWF0aWNzTGFiQXRNVU4vc1JOQVJhbmtpbmcpCiogT3JpZ1JGX1tzY2FsZWQvbm9ybWFsaXplZF0gPSByZWZlcnMgdG8gYW55IG1vZGVsIHRoYXQgd2FzIHRyYWluZWQgaW4gdGhlIHNhbWUgbWFubmVyKHNhbWUgbGlicmFyaWVzIGFuZCBzZXR0aW5ncykgYXMgaW4gdGhlIE9yaWdpbmFsIHJhbmRvbSBmb3Jlc3QgbW9kZWwsIGJ1dCB1c2luZyBzY2FsZWQgb3Igbm9ybWFsaXplZCBkYXRhLiAKICAqICBCeSBzY2FsaW5nIHRoZSBkYXRhIHdlIG1lYW4gc2NhbGluZyBhbmQgY2VudGVyaW5nCiAgKiAgQnkgbm9ybWFsaXppbmcgd2UgbWVhbiBmaXQgdGhlIHZhbHVlcyBiZXR3ZWVuIDAgYW5kIDEKKiBSRkUgPSBSYW5kb20gRm9yZXN0IEV4cGxhaW5lcgoqIFJGID0gUmFuZG9tIEZvcmVzdAoqIElNID0gSW50ZXJwcmV0YWJpbGl0eSBNb2RlbAoqICJTUyIgPSB0aGUgZnJlZSBlbmVyZ3kgb2YgdGhlIHByZWRpY3RlZCBzZWNvbmRhcnkgc3RydWN0dXJlIG9mIHRoZSBzUk5BIC0gbW9zdGx5IG5lZ2F0aXZlIHZhbHVlcwoqICJQb3MxMHdydHNSTkFTdGFydCIgPSBkaXN0YW5jZSB0byB0aGVpciBjbG9zZXN0IHByZWRpY3RlZCBwcm9tb3RlciBzaXRlCiogIkRpc3RUZXJtIiA9IGRpc3RhbmNlIHRvIHRoZWlyIGNsb3Nlc3QgcHJlZGljdGVkIFJoby1pbmRlcGVuZGVudCB0ZXJtaW5hdG9yIAoqICJEaXN0YW5jZSIgPSBkaXN0YW5jZSB0byB0aGUgY2xvc2VzdCByZWFkaW5nIGZyYW1lIG9uIHRoZSBMRUZUKCJ1cHN0cmVhbSIpIHNpZGUKKiAic2FtZVN0cmFuZCIgPSBib29sZWFuLCBpZiB0cmFuc2NyaXB0aW9uIGlzIGdvaW5nIGluIHRoZSBzYW1lIGRpcmVjdGlvbiBhcyB0aGUgT1JGKExlZnQgT3BlbiBSZWFkaW5nIEZyYW1lKQogICogIE9SRiA9IGdlbm9taWMgc2VxdWVuY2UgdGhhdCdzIHN1cHBvc2VkIHRvIGNvZGUgZm9yIGEgcHJvdGVpbgoqICJEb3duRGlzdGFuY2UiICA9IGRpc3RhbmNlIHRvIHRoZSBjbG9zZXN0IHJlYWRpbmcgZnJhbWUgb24gdGhlIFJJR0hUKCJkb3duc3RyZWFtIikgc2lkZQoqICJzYW1lRG93blN0cmFuZCIgPSAgYm9vbGVhbiwgaWYgdHJhbnNjcmlwdGlvbiBpcyBnb2luZyBpbiB0aGUgc2FtZSBkaXJlY3Rpb24gYXMgdGhlIFJPUkYoUmlnaHQgT3BlbiBSZWFkaW5nIEZyYW1lKQoKKipTaWRlIG5vdGU6KioqIFdoZW4gdXNlZCB0aGUgd29yZCBvcmlnaW5hbCB0byByZWZlciB0byB0aGUgUkYgbW9kZWwgY3JlYXRlZCBpbiB0aGUgYmFzZSBhcnRpY2xlLCBvciB0byByZWZlciB0byBtb2RlbHMgdGhhdCB3ZXJlIGNyZWF0ZWQodHJhaW5lZCkgaW4gdGhlIHNhbWUgd2F5IGFzIHRoZSBiYXNlIHJhbmRvbSBmb3Jlc3QgbW9kZWwuIAoKIyBJKSBQUkUtU0VUVVAKVGhpcyBzZWN0aW9uIGlzIGFsbCBhYm91dCBnZXR0aW5nIHRoZSBsaWJyYXJpZXMgYW5kIGRhdGEgbmVlZGVkIHRvIHJ1biB0aGlzIFIgbm90ZWJvb2suIFdlIHdpbGwgcHJlcHJvY2VzcyB0aGUgZGF0YSBhbmQgbG9hZCB0aGUgb3JpZ2luYWwgbW9kZWwuIFdlIHN0YXJ0IHdpdGggc29tZSAqKk9QVElPTkFMKiogbWVtb3J5IGNsZWFuaW5nICh1c2VmdWwgaWYgeW91IHdhbnQgdG8gcnVuIHRoaXMgZnJvbSBSIFN0dWRpbyBhbmQgd2l0aCBhIGNsZWFuIGVudmlyb25tZW50KQpgYGB7cn0KIyBUaGlzIHNlY3Rpb24gaXMganVzdCBhYm91dCBnZXR0aW5nIHRoZSBlbnZpcm9ubWVudCByZWFkeSBhbmQgY2xlYW5lZCB1cC4gU2luY2UgdGhpcyBwcm9qZWN0IHdhcyBvcmlnaW5hbGx5IHdyaXR0ZW4gYXMgYSByZWd1bGFyIFIgc2NyaXB0LCBhbmQgd29ya2VkIGZyb20gd2l0aGluIFIgU3R1ZGlvLCBpdCB3YXMganVzdCB1c2VmdWxsIGluIHJlc2V0dGluZyBhbmQgY2xlYW5pbmcgdXAgdGhlIHRoZSBlbnZpcm9ubWVudCwgYW5kIHNvIHRoaXMgYmxvY2sgb2YgY29kZSBpcyBvcHRpb25hbApjbGMgPC0gZnVuY3Rpb24oKSBjYXQoIlwwMTQiKSA7IGNsYygpCnJlbW92ZShsaXN0ID0gbHMoKSkKZ2MoKQpgYGAKCiMjIDEuIEluc3RhbGwgcmVxdWlyZWQgbGlicmFyaWVzIAoKVGhlIGZvbGxvd2luZyBjb2RlIGlzIGJhc2VkIG9uIHRoZSBoMm8gZG9jdW1lbnRhdGlvbiBzaXRlLCBhcyBpdCdzIHRoZWlyIHJlY29tbWVuZGVkIHdheSBmb3IgZG93bmxvYWRpbmcgYW5kIGluc3RhbGxpbmcgaDJvIGZvciBSLCB3aXRoIG9ubHkgdGhlIGZpcnN0IGlmIGJlaW5nIGFkZGVkIGFzIHRvIHByZXZlbnQgYWNjaWRlbnRhbCByZWluc3RhbGxhdGlvbnMuIEhlcmUgaXMgdGhlIFtsaW5rXShodHRwOi8vaDJvLXJlbGVhc2UuczMuYW1hem9uYXdzLmNvbS9oMm8vcmVsLXphaHJhZG5pay80L2luZGV4Lmh0bWwpIGZvciB0aG9zZSBpbnRlcmVzdGVkIGluIGl0LiAgCgpgYGB7cn0KCmlmICghICgiaDJvIiAlaW4lIHJvd25hbWVzKGluc3RhbGxlZC5wYWNrYWdlcygpKSkpIHsgCiAgCiAgICAjIFRoZSBmb2xsb3dpbmcgdHdvIGNvbW1hbmRzIHJlbW92ZSBhbnkgcHJldmlvdXNseSBpbnN0YWxsZWQgSDJPIHBhY2thZ2VzIGZvciBSLgogICAgaWYgKCJwYWNrYWdlOmgybyIgJWluJSBzZWFyY2goKSkgeyBkZXRhY2goInBhY2thZ2U6aDJvIiwgdW5sb2FkPVRSVUUpIH0KICAgIGlmICgiaDJvIiAlaW4lIHJvd25hbWVzKGluc3RhbGxlZC5wYWNrYWdlcygpKSkgeyByZW1vdmUucGFja2FnZXMoImgybyIpIH0KICAgICMgTmV4dCwgd2UgZG93bmxvYWQgcGFja2FnZXMgdGhhdCBIMk8gZGVwZW5kcyBvbi4KICAgIHBrZ3MgPC0gYygiUkN1cmwiLCJqc29ubGl0ZSIpCiAgICBmb3IgKHBrZyBpbiBwa2dzKSB7CiAgICAgIGlmICghIChwa2cgJWluJSByb3duYW1lcyhpbnN0YWxsZWQucGFja2FnZXMoKSkpKSB7IGluc3RhbGwucGFja2FnZXMocGtnKSB9CiAgICB9CiAgICAjIE5vdyB3ZSBkb3dubG9hZCwgaW5zdGFsbCBhbmQgaW5pdGlhbGl6ZSB0aGUgSDJPIHBhY2thZ2UgZm9yIFIuCiAgICBpbnN0YWxsLnBhY2thZ2VzKCJoMm8iLCB0eXBlPSJzb3VyY2UiLCByZXBvcz0iaHR0cDovL2gyby1yZWxlYXNlLnMzLmFtYXpvbmF3cy5jb20vaDJvL3JlbC16YWhyYWRuaWsvNC9SIikKfSAgCmBgYAoKVGhlIHJlc3Qgb2YgdGhlIGxpYnJhcmllcyBjYW4gYmUgaW5zdGFsbGVkIGluIHRoZSBzYW1lIHdheSBhcyB3aXRoIGFueSBvdGhlciBsaWJyYXJ5LgpgYGB7cn0KcGFja2FnZXMgPC0gYygibGltZSIsICJST0NSIiwgIlBSUk9DIiwgInJhbmRvbUZvcmVzdCIsICJyYW5kb21Gb3Jlc3RFeHBsYWluZXIiLCAicG5nIikKZm9yIChwYWNrYWdlIGluIHBhY2thZ2VzKSB7CiAgaWYgKCEgKHBhY2thZ2UgJWluJSByb3duYW1lcyhpbnN0YWxsZWQucGFja2FnZXMoKSkpKSB7IGluc3RhbGwucGFja2FnZXMocGFja2FnZSkgfQp9CgpgYGAKCiMjIDIuIExvYWQgdGhlIGxpYnJhcmllcyAKCmBgYHtyfQpsaWJyYXJ5KCJoMm8iKSAgICAgICAgICAjIE1MIG1vZGVsIGJ1aWxkaW5nCmxpYnJhcnkoIlJPQ1IiKSAgICAgICAgICMgTUwgZXZhbHVhdGlvbgpsaWJyYXJ5KCJQUlJPQyIpICAgICAgICAjIE1MIGV2YWx1YXRpb24KbGlicmFyeSgicmFuZG9tRm9yZXN0IikgIyBNTCBtb2RlbCBidWlsZGluZwpsaWJyYXJ5KCJyYW5kb21Gb3Jlc3RFeHBsYWluZXIiKSAjIE1MIEdsb2JhbCBpbnRlcnByZXRhdGlvbgpsaWJyYXJ5KCJsaW1lIikgICAgICAgICAjIE1MIGxvY2FsIGludGVycHJldGF0aW9uCmxpYnJhcnkoInBuZyIpCmBgYAoKIyMgMy4gTG9hZCBhbmQgcHJlcHJvY2VzcyB0aGUgZGF0YSAKCiMjIyAzLjEgTG9hZCB0aGUgTW9kZWwncyBUcmFpbmluZyBEYXRhIApgYGB7cn0KdHJhaW5EYXRhU2V0IDwtIHJlYWQuY3N2KCIuL0RhdGFTZXRzL2NvbWJpbmVkRGF0YS5jc3YiLCBoZWFkZXIgPSBUUlVFKQp0cmFpbkRhdGFTZXRbLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwodHJhaW5EYXRhU2V0WywiQ2xhc3MiXSkgIyBndWFyYW50ZWVzIHRoYXQgb3VyIGgybyBtb2RlbCB0cmFpbnMgdGhlIHJhbmRvbSBmb3Jlc3QgKFJGKSBhcyBhIGNsYXNzaWZpY2F0aW9uIHRyZWUgYW5kIG5vdCBhIHJlZ3Jlc3Npb24gdHJlZQpgYGAKCiMjIyAzLjIgU2NhbGUgYW5kIENlbnRlciB0aGUgVHJhaW5pbmcgRGF0YSAKCmBgYHtyfQpkYXRhU2V0VHJhaW5fc2NhbGVkIDwtIHNjYWxlKHRyYWluRGF0YVNldFssLWMoNSw3LDgsOSldLCBjZW50ZXIgPSBUUlVFLCBzY2FsZSA9IFRSVUUpCmRhdGFTZXRUcmFpbl9zY2FsZWQgPC0gY2JpbmQoIGRhdGFTZXRUcmFpbl9zY2FsZWRbLCgxOjQpXSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YVNldFssNV0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGFTZXRUcmFpbl9zY2FsZWRbLDVdLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFTZXRbLGMoNyw4LDkpXSkKY29sbmFtZXMoZGF0YVNldFRyYWluX3NjYWxlZCkgPC0gYygiU1MiLCAiUG9zMTB3cnRzUk5BU3RhcnQiLCAiRGlzdFRlcm0iLCAiRGlzdGFuY2UiLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAic2FtZVN0cmFuZCIsICJEb3duRGlzdGFuY2UiLCAic2FtZURvd25TdHJhbmQiLCAiSUQiLCAiQ2xhc3MiKQpkYXRhU2V0VHJhaW5fc2NhbGVkWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKGRhdGFTZXRUcmFpbl9zY2FsZWRbLCJDbGFzcyJdKSAjIEp1c3QgbWFraW5nIHN1cmUgQ2xhc3MgaXMgdGFrZW4gYSBsb2dpY2FsIHZhbHVlCnJiaW5kKCBoZWFkKGRhdGFTZXRUcmFpbl9zY2FsZWQsNSksIHRhaWwoZGF0YVNldFRyYWluX3NjYWxlZCw1KSApCgpgYGAKCiMjIyAzLjMgTm9ybWFsaXplIHRoZSBUcmFpbmluZyBEYXRhCmBgYHtyfQpub3JtYWxpemUgPC0gZnVuY3Rpb24oeCkgewogIHJldHVybiAoKHggLSBtaW4oeCkpIC8gKG1heCh4KSAtIG1pbih4KSkpCn0KCmRhdGFTZXRUcmFpbl9ub3JtIDwtIHRyYWluRGF0YVNldApkYXRhU2V0VHJhaW5fbm9ybSRTUyA8LSBub3JtYWxpemUoZGF0YVNldFRyYWluX25vcm0kU1MpCmRhdGFTZXRUcmFpbl9ub3JtJFBvczEwd3J0c1JOQVN0YXJ0IDwtIG5vcm1hbGl6ZShkYXRhU2V0VHJhaW5fbm9ybSRQb3MxMHdydHNSTkFTdGFydCkKZGF0YVNldFRyYWluX25vcm0kRGlzdFRlcm0gPC0gbm9ybWFsaXplKGRhdGFTZXRUcmFpbl9ub3JtJERpc3RUZXJtKQpkYXRhU2V0VHJhaW5fbm9ybSREaXN0YW5jZSA8LSBub3JtYWxpemUoZGF0YVNldFRyYWluX25vcm0kRGlzdGFuY2UpCmRhdGFTZXRUcmFpbl9ub3JtJERvd25EaXN0YW5jZSA8LSBub3JtYWxpemUoZGF0YVNldFRyYWluX25vcm0kRG93bkRpc3RhbmNlKQpkYXRhU2V0VHJhaW5fbm9ybVssIkNsYXNzIl0gPC0gYXMubG9naWNhbChkYXRhU2V0VHJhaW5fbm9ybVssIkNsYXNzIl0pICMgSnVzdCBtYWtpbmcgc3VyZSBDbGFzcyBpcyB0YWtlbiBhIGxvZ2ljYWwgdmFsdWUKcmJpbmQoIGhlYWQoZGF0YVNldFRyYWluX25vcm0sNSksIHRhaWwoZGF0YVNldFRyYWluX25vcm0sNSkgKSAKYGBgCgojIyMgMy40IENoZWNrIGZvciBub3JtYWxpdHkgaW4gdGhlIGRhdGEKVGhlcmUgYXJlIGNlcnRhaW4gcGFyYW1ldGVyKCkgaW4gdGhlIExJTUUgZnVuY3Rpb25zIHRoYXQgYXJlIHJlY29tbWVuZGVkIGJhc2VkIG9uIHRoZSBkaXN0cmlidXRpb24gb2YgdGhlIGRhdGEuIEJhc2VkIG9uIHRoZXNlIHBhcmVtZXRlcnMsIHdlIGRlY2lkZWQgdG8gY2hlY2sgZm9yIG5vcm1hbGl0eSBpbiB0aGUgZGF0YQoKYGBge3J9CmRhdGFTZXRUcmFpblggPC0gdHJhaW5EYXRhU2V0WywtKDg6OSldCmRhdGFTZXRUcmFpblkgPC0gdHJhaW5EYXRhU2V0WywoODo5KV0gIAoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywxXSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIFNTIikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssMV0pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywxXSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywyXSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIFBvczEwd3J0c1JOQVN0YXJ0IikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssMl0pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywyXSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywzXSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIERpc3RUZXJtIikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssM10pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywzXSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw0XSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIERpc3RhbmNlIikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssNF0pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw0XSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw1XSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIHNhbWVTdHJhbmQiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw1XSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDVdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDZdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2YgRG93bkRpc3RhbmNlIikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssNl0pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw2XSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQoKaGlzdCh4ID0gYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw3XSksIG1haW4gPSAiSGlzdG9ncmFtIG9mIHNhbWVEb3duU3RyYW5kIikKcXFub3JtKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssN10pKQpxcWxpbmUoYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw3XSksIGNvbCA9ICJyZWQiLCBsd2QgPSAyKQpgYGAKCiMjIyAzLjUgTG9hZCB0ZXN0aW5nIERhdGFzZXRzCkxvYWQgT3JpZ2luYWwgVGVzdGluZyBEYXRhIFNldHMKCmBgYHtyfQpzbHQyZGF0YVBvcyA8LSByZWFkLmNzdigiLi9EYXRhU2V0cy9TTFQyX1Bvc2l0aXZlcy50c3YiLCBzZXAgPSAiXHQiLCBoZWFkZXIgPSBUUlVFKQpzbHQyZGF0YVBvcyRDbGFzcyA8LSByZXAoMSxucm93KHNsdDJkYXRhUG9zKSkKc2x0MmRhdGFOZWcgPC0gcmVhZC5jc3YoIi4vRGF0YVNldHMvU0xUMl9OZWdhdGl2ZXMudHN2Iiwgc2VwID0gIlx0IiwgaGVhZGVyID0gVFJVRSkKc2x0MmRhdGFOZWckQ2xhc3MgPC0gcmVwKDAsbnJvdyhzbHQyZGF0YU5lZykpCnNsdDJkYXRhIDwtIHJiaW5kKHNsdDJkYXRhUG9zLHNsdDJkYXRhTmVnKQoKbHVkYXRhUG9zIDwtIHJlYWQuY3N2KCIuL0RhdGFTZXRzL0x1X1Bvc2l0aXZlcy50c3YiLCBzZXAgPSAiXHQiLCBoZWFkZXIgPSBUUlVFKQpsdWRhdGFQb3MkQ2xhc3MgPC0gcmVwKDEsbnJvdyhsdWRhdGFQb3MpKQpsdWRhdGFOZWcgPC0gcmVhZC5jc3YoIi4vRGF0YVNldHMvTHVfTmVnYXRpdmVzLnRzdiIsIHNlcCA9ICJcdCIsIGhlYWRlciA9IFRSVUUpCmx1ZGF0YU5lZyRDbGFzcyA8LSByZXAoMCxucm93KGx1ZGF0YU5lZykpCmx1ZGF0YSA8LSByYmluZChsdWRhdGFQb3MsbHVkYXRhTmVnKQpgYGAKCiMjIyAzLjYgU2NhbGUgdGVzdGluZyBEYXRhc2V0cwpFdmVuIHRob3VnaCB0aGUgKnNjYWxlIGZ1bmN0aW9uKiBjYW4gc2NhbGUgYW5kIGNlbnRlciB0aGUgZGF0YSBmb3IgdXMsIHdlIG5lZWQgdG8gc2NhbGUgdGhlIHRlc3RpbmcgZGF0YSBzZXRzIHVzaW5nIHRoZSB0cmFpbmluZyBkYXRhJ3Mgc3RhbmRhcmQgZGV2aWF0aW9uIGFuZCBtZWFucyB0byBvYnRhaW4gYWRlcXVhdGUgcmVzdWx0cy4KClNjYWxpbmcgRnVuY3Rpb246CiogWGkgICAgPSBBbiBpbnN0YW5jZSAKKiBYbWVhbiA9IE1lYW4vQXZlcmFnZSBvZiBhbGwgc2FtcGxlcyAgCiogWHNkICAgPSBTdGFuZGFyZCBEZXZpYXRpb24gb2Ygb3VyIGRhdGEgc2V0IAoqIFppICAgID0gU2NhbGVkIFZhbHVlIG9mIFhpCiogWmkgICAgPSAoWGkgLSBYbWVhbikgLyBYc2QgCgpgYGB7cn0KIyBTdGFuZGFyZCBEZXZpYXRpb24gYW5kIE1lYW4gZm9yIFN0YW5kYXJpemF0aW9uCnNjYWxlX3RkIDwtIGZ1bmN0aW9uKCBYaSwgWG1lYW4sIFhzZCkgewogIHJldHVybiAoIChYaSAtIFhtZWFuKSAvIFhzZCApICMgWmkKfQp0cmFpbkRhdGFfc2RfU1MgICAgICAgIDwtIHNkKHRyYWluRGF0YVNldFssIlNTIl0pCnRyYWluRGF0YV9zZF9Qb3MgICAgICAgPC0gc2QodHJhaW5EYXRhU2V0WywiUG9zMTB3cnRzUk5BU3RhcnQiXSkKdHJhaW5EYXRhX3NkX0Rpc1Rlcm0gICA8LSBzZCh0cmFpbkRhdGFTZXRbLCJEaXN0VGVybSJdKQp0cmFpbkRhdGFfc2RfRGlzICAgICAgIDwtIHNkKHRyYWluRGF0YVNldFssIkRpc3RhbmNlIl0pCnRyYWluRGF0YV9zZF9Eb3duRGlzICAgPC0gc2QodHJhaW5EYXRhU2V0WywiRG93bkRpc3RhbmNlIl0pCgp0cmFpbkRhdGFfbWVhbl9TUyAgICAgIDwtIG1lYW4odHJhaW5EYXRhU2V0WywiU1MiXSkKdHJhaW5EYXRhX21lYW5fUG9zICAgICA8LSBtZWFuKHRyYWluRGF0YVNldFssIlBvczEwd3J0c1JOQVN0YXJ0Il0pCnRyYWluRGF0YV9tZWFuX0Rpc1Rlcm0gPC0gbWVhbih0cmFpbkRhdGFTZXRbLCJEaXN0VGVybSJdKQp0cmFpbkRhdGFfbWVhbl9EaXMgICAgIDwtIG1lYW4odHJhaW5EYXRhU2V0WywiRGlzdGFuY2UiXSkKdHJhaW5EYXRhX21lYW5fRG93bkRpcyA8LSBtZWFuKHRyYWluRGF0YVNldFssIkRvd25EaXN0YW5jZSJdKQoKc2x0MmRhdGFfc2NhbGVkIDwtIHNsdDJkYXRhCnNsdDJkYXRhX3NjYWxlZCRTUyAgICAgICAgICAgICAgICA8LSBzY2FsZV90ZChzbHQyZGF0YV9zY2FsZWQkU1MsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21lYW5fU1MsIHRyYWluRGF0YV9zZF9TUykKc2x0MmRhdGFfc2NhbGVkJFBvczEwd3J0c1JOQVN0YXJ0IDwtIHNjYWxlX3RkKHNsdDJkYXRhX3NjYWxlZCRQb3MxMHdydHNSTkFTdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX1BvcywgdHJhaW5EYXRhX3NkX1BvcykKc2x0MmRhdGFfc2NhbGVkJERpc3RUZXJtICAgICAgICAgIDwtIHNjYWxlX3RkKHNsdDJkYXRhX3NjYWxlZCREaXN0VGVybSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9EaXNUZXJtLCB0cmFpbkRhdGFfc2RfRGlzVGVybSkKc2x0MmRhdGFfc2NhbGVkJERpc3RhbmNlICAgICAgICAgIDwtIHNjYWxlX3RkKHNsdDJkYXRhX3NjYWxlZCREaXN0YW5jZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX0RpcywgdHJhaW5EYXRhX3NkX0RpcykKc2x0MmRhdGFfc2NhbGVkJERvd25EaXN0YW5jZSAgICAgIDwtIHNjYWxlX3RkKHNsdDJkYXRhX3NjYWxlZCREb3duRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9Eb3duRGlzLCB0cmFpbkRhdGFfc2RfRG93bkRpcykKbHVkYXRhX3NjYWxlZCA8LSBsdWRhdGEKCmx1ZGF0YV9zY2FsZWQkU1MgICAgICAgICAgICAgICAgPC0gc2NhbGVfdGQobHVkYXRhX3NjYWxlZCRTUywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9TUywgdHJhaW5EYXRhX3NkX1NTKQpsdWRhdGFfc2NhbGVkJFBvczEwd3J0c1JOQVN0YXJ0IDwtIHNjYWxlX3RkKGx1ZGF0YV9zY2FsZWQkUG9zMTB3cnRzUk5BU3RhcnQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9Qb3MsIHRyYWluRGF0YV9zZF9Qb3MpCmx1ZGF0YV9zY2FsZWQkRGlzdFRlcm0gICAgICAgICAgPC0gc2NhbGVfdGQobHVkYXRhX3NjYWxlZCREaXN0VGVybSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9EaXNUZXJtLCB0cmFpbkRhdGFfc2RfRGlzVGVybSkKbHVkYXRhX3NjYWxlZCREaXN0YW5jZSAgICAgICAgICA8LSBzY2FsZV90ZChsdWRhdGFfc2NhbGVkJERpc3RhbmNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21lYW5fRGlzLCB0cmFpbkRhdGFfc2RfRGlzKQpsdWRhdGFfc2NhbGVkJERvd25EaXN0YW5jZSAgICAgIDwtIHNjYWxlX3RkKGx1ZGF0YV9zY2FsZWQkRG93bkRpc3RhbmNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21lYW5fRG93bkRpcywgdHJhaW5EYXRhX3NkX0Rvd25EaXMpCgpyYmluZCggaGVhZChzbHQyZGF0YV9zY2FsZWQsNSksIHRhaWwoc2x0MmRhdGFfc2NhbGVkLDUpKQpyYmluZCggaGVhZChsdWRhdGFfc2NhbGVkLDUpLCB0YWlsKGx1ZGF0YV9zY2FsZWQsNSkpCgpgYGAKCiMjIyAzLjcgTm9ybWFsaXplIHRlc3RpbmcgRGF0YXNldHMKSW4gYSBzaW1pbGFyIGZhc2hpb24gdG8gd2hhdCB3ZSBkaWQgaW4gMy42LCB3ZSBhbHNvIG5lZWQgdG8gdXNlIHRoZSB0cmFpbmluZyBkYXRhJ3MgbWF4IGFuZCBtaW4gdmFsdWVzIHRvIHByb3Blcmx5IG5vcm1hbGl6ZSBlYWNoIGZlYXR1cmUuCgpgYGB7cn0Kbm9ybWFsaXplX3RkIDwtIGZ1bmN0aW9uKHgsIG1pbiwgbWF4KSB7CiAgcmV0dXJuICggKHggLSBtaW4pIC8gKG1heCAtIG1pbikgKQp9Cgp0cmFpbkRhdGFfbWF4X1NTICAgICAgPC0gbWF4KHRyYWluRGF0YVNldFssIlNTIl0pCnRyYWluRGF0YV9tYXhfUG9zICAgICA8LSBtYXgodHJhaW5EYXRhU2V0WywiUG9zMTB3cnRzUk5BU3RhcnQiXSkKdHJhaW5EYXRhX21heF9EaXNUZXJtIDwtIG1heCh0cmFpbkRhdGFTZXRbLCJEaXN0VGVybSJdKQp0cmFpbkRhdGFfbWF4X0RpcyAgICAgPC0gbWF4KHRyYWluRGF0YVNldFssIkRpc3RhbmNlIl0pCnRyYWluRGF0YV9tYXhfRG93bkRpcyA8LSBtYXgodHJhaW5EYXRhU2V0WywiRG93bkRpc3RhbmNlIl0pCgp0cmFpbkRhdGFfbWluX1NTICAgICAgPC0gbWluKHRyYWluRGF0YVNldFssIlNTIl0pCnRyYWluRGF0YV9taW5fUG9zICAgICA8LSBtaW4odHJhaW5EYXRhU2V0WywiUG9zMTB3cnRzUk5BU3RhcnQiXSkKdHJhaW5EYXRhX21pbl9EaXNUZXJtIDwtIG1pbih0cmFpbkRhdGFTZXRbLCJEaXN0VGVybSJdKQp0cmFpbkRhdGFfbWluX0RpcyAgICAgPC0gbWluKHRyYWluRGF0YVNldFssIkRpc3RhbmNlIl0pCnRyYWluRGF0YV9taW5fRG93bkRpcyA8LSBtaW4odHJhaW5EYXRhU2V0WywiRG93bkRpc3RhbmNlIl0pCgpzbHQyZGF0YV9ub3JtIDwtIHNsdDJkYXRhCnNsdDJkYXRhX25vcm0kU1MgICAgICAgICAgICAgICAgPC0gbm9ybWFsaXplX3RkKHNsdDJkYXRhX25vcm0kU1MsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21pbl9TUywgdHJhaW5EYXRhX21heF9TUykKc2x0MmRhdGFfbm9ybSRQb3MxMHdydHNSTkFTdGFydCA8LSBub3JtYWxpemVfdGQoc2x0MmRhdGFfbm9ybSRQb3MxMHdydHNSTkFTdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fUG9zLCB0cmFpbkRhdGFfbWF4X1BvcykKc2x0MmRhdGFfbm9ybSREaXN0VGVybSAgICAgICAgICA8LSBub3JtYWxpemVfdGQoc2x0MmRhdGFfbm9ybSREaXN0VGVybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fRGlzVGVybSwgdHJhaW5EYXRhX21heF9EaXNUZXJtKQpzbHQyZGF0YV9ub3JtJERpc3RhbmNlICAgICAgICAgIDwtIG5vcm1hbGl6ZV90ZChzbHQyZGF0YV9ub3JtJERpc3RhbmNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21pbl9EaXMsIHRyYWluRGF0YV9tYXhfRGlzKQpzbHQyZGF0YV9ub3JtJERvd25EaXN0YW5jZSAgICAgIDwtIG5vcm1hbGl6ZV90ZChzbHQyZGF0YV9ub3JtJERvd25EaXN0YW5jZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX0Rvd25EaXMsIHRyYWluRGF0YV9tYXhfRG93bkRpcykKbHVkYXRhX25vcm0gPC0gbHVkYXRhCmx1ZGF0YV9ub3JtJFNTICAgICAgICAgICAgICAgIDwtIG5vcm1hbGl6ZV90ZChsdWRhdGFfbm9ybSRTUywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX1NTLCB0cmFpbkRhdGFfbWF4X1NTKQpsdWRhdGFfbm9ybSRQb3MxMHdydHNSTkFTdGFydCA8LSBub3JtYWxpemVfdGQobHVkYXRhX25vcm0kUG9zMTB3cnRzUk5BU3RhcnQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX1BvcywgdHJhaW5EYXRhX21heF9Qb3MpCmx1ZGF0YV9ub3JtJERpc3RUZXJtICAgICAgICAgIDwtIG5vcm1hbGl6ZV90ZChsdWRhdGFfbm9ybSREaXN0VGVybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fRGlzVGVybSwgdHJhaW5EYXRhX21heF9EaXNUZXJtKQpsdWRhdGFfbm9ybSREaXN0YW5jZSAgICAgICAgICA8LSBub3JtYWxpemVfdGQobHVkYXRhX25vcm0kRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX0RpcywgdHJhaW5EYXRhX21heF9EaXMpCmx1ZGF0YV9ub3JtJERvd25EaXN0YW5jZSAgICAgIDwtIG5vcm1hbGl6ZV90ZChsdWRhdGFfbm9ybSREb3duRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX0Rvd25EaXMsIHRyYWluRGF0YV9tYXhfRG93bkRpcykKCnJiaW5kKCBoZWFkKHNsdDJkYXRhX25vcm0sNSksIHRhaWwoc2x0MmRhdGFfbm9ybSw1KSkKCnJiaW5kKCBoZWFkKGx1ZGF0YV9ub3JtLDUpLCB0YWlsKGx1ZGF0YV9ub3JtLDUpKQoKYGBgCgojIyA0LiBMb2FkIG9yaWdpbmFsIG1vZGVsCgpUaGUgb3JpZ2luYWwgYXJ0aWNsZSwgdGl0bGVkICJQcmlvcml0aXppbmcgYm9uYSBmaWRlIGJhY3RlcmlhbCBzbWFsbCBSTkFzIHdpdGggbWFjaGluZSBsZWFybmluZyBjbGFzc2lmaWVycyIsIGFuZCB0aGUgc291cmNlIG1hdGVyaWFscyh0cmFpbmluZyBkYXRhLCB0ZXN0aW5nIGRhdGEsIFIgc2NyaXB0cywgLnJkcyBmaWxlLCBldGMuKSBjYW4gYWxsIGJlIGZvdW5kIGluIFtQZWVySl0oaHR0cHM6Ly9wZWVyai5jb20vYXJ0aWNsZXMvNjMwNC8pIHVuZGVyIHRoZSBmb2xsb3dpbmcgW2xpbmtdKGh0dHBzOi8vcGVlcmouY29tL2FydGljbGVzLzYzMDQvKS4KVGhlIG9yaWdpbmFsICoucmRzKiBmaWxlIGNhbiBhbHNvIGJlIGRvd25sb2FkZWQgZnJvbSBbZ2l0aHViXShodHRwczovL2dpdGh1Yi5jb20vQmlvaW5mb3JtYXRpY3NMYWJBdE1VTi9zUk5BUmFua2luZykuCgojIyMgNC4xIExvYWQvUmV0cmFpbiBPcmlnIE1vZGVsCldlIGNhbiBlaXRoZXIgcmV0cmFpbiB0aGUgb3JpZ2luYWwgbW9kZWwsIG9yIGxvYWQgdGhlIC5yZHMgZmlsZSBwcm92aWRlZCBpbiB0aGUgb3JpZ2luYWwgYXJ0aWNsZS4gRm9yIHRoZSBwdXJwb3NlIG9mIHNhdmluZyB0aW1lLCB3ZSBzaW1wbHkgbG9hZGVkIHRoZSBtb2RlbC4KCmBgYHtyfQpvcmlnUkYgPC0gcmVhZFJEUygiUkZfY2xhc3NpZmllcjRzUk5BLnJkcyIpCmBgYAoKIyMjIDQuMiBSZXRyYWluIG5ldyBtb2RlbHMgd2l0aCBTY2FsZWQgYW5kIE5vcm1hbGl6ZWQgRGF0YSAKU2luY2Ugc29tZSBvZiB0aGUgSU0gbW9kZWxzIHJlcXVpcmVkIHNjYWxpbmcgb3Igbm9ybWFsaXplZCBkYXRhIHRvIHByb3Blcmx5IGZ1bmN0aW9uLCB3ZSBuZWVkZWQgdG8gY3JlYXRlIDIgbmV3IG1vZGVscyBpbiB0aGUgc2FtZSB3YXkgYXMgdGhlIG9uZSBsb2FkZWQgaW4gNC4xLiAKCmBgYHtyfQp0dW5lUkYoZGF0YVNldFRyYWluWFssYygxOjcpXSwgeSA9IGZhY3RvcihkYXRhU2V0VHJhaW5ZWywyXSksIG50cmVlVHJ5ID0gNDAwLCBtdHJ5U3RhcnQgPSAyKQpzZXQuc2VlZCgxMjM0KQpvcmlnUkZfc2NhbGVkIDwtIHJhbmRvbUZvcmVzdCh4ID0gZGF0YVNldFRyYWluX3NjYWxlZFssYygxOjcpXSwgeSA9IGZhY3RvcihkYXRhU2V0VHJhaW5ZWywyXSksIG10cnkgPSAyLCBudHJlZSA9IDQwMCwgaW1wb3J0YW5jZSA9IFRSVUUpCm9yaWdSRl9ub3JtIDwtIHJhbmRvbUZvcmVzdCh4ID0gZGF0YVNldFRyYWluX25vcm1bLGMoMTo3KV0sIHkgPSBmYWN0b3IoZGF0YVNldFRyYWluWVssMl0pLCBtdHJ5ID0gMiwgbnRyZWUgPSA0MDAsIGltcG9ydGFuY2UgPSBUUlVFKQpgYGAKCiMgSUkpIFRSQUlOIFRIRSBORVcgSDJPIE1PREVMIApTaW5jZSB3ZSBhcmUgZ29pbmcgdG8gdXNlIHRoZSBoMm8gbGlicmFyeSB0byBwZXJmb3JtIHNvbWUgb2Ygb3VyIG1hY2hpbmUgbGVhcm5pbmcgaW50ZXJwcmV0YWJpbGl0eSwgd2UgbmVlZCB0byB0cmFpbiBhbiBlcXVpdmFsZW50IG9uZSwgdXNpbmcgdGhlIGgybyB0b29scy4KCiMjIDUuIEludGlhbGl6ZSBIMk8gYW5kIGxvYWQgdHJhaW5pbmcgZGF0YSAKCmBgYHtyfQpoMm8uaW5pdCggICAgICAgICAgICAgICMgSW5pdGlhbGl6ZSBoMm8KICBudGhyZWFkcyA9IC0xLCAgICAgICAjIC0xID0gdXNlIGFsbCBhdmFpbGFibGUgdGhyZWFkcwogIG1heF9tZW1fc2l6ZSA9ICI4RyIgICMgc3BlY2lmeSB0aGUgYW1vdW50IG9mIG1lbW9yeSB0byB1c2UKKQpoMm8ucmVtb3ZlQWxsKCkgICAgICAgICMgY2xlYW4gdXAgdGhlIHN5c3RlbSwgaDJvLXdpc2UgKGllLiBraWxsIGFueSBydW5uaW5nIGgybyBjbHVzdGVycykKaDJvLm5vX3Byb2dyZXNzKCkgCnRyYWluRGF0YSA8LSBhcy5oMm8odHJhaW5EYXRhU2V0KSAKdHJhaW5EYXRhX3NjYWxlZCA8LSBhcy5oMm8oZGF0YVNldFRyYWluX3NjYWxlZCkgCnRyYWluRGF0YV9ub3JtIDwtIGFzLmgybyhkYXRhU2V0VHJhaW5fbm9ybSkgCmBgYAoKIyMgNi4gQnVpbGQgbW9kZWwKIyMjIDYuMSBCdWlsZCBtb2RlbCB3aXRoIE9yaWcgVHJhaW5pbmcgZGF0YQpgYGB7cn0KcmZoMm8gPC0gaDJvLnJhbmRvbUZvcmVzdCggCiAgICAgICAgICAgICB0cmFpbmluZ19mcmFtZSA9IHRyYWluRGF0YSwgIyB0cmFpbmluZyB1c2luZyB0aGUgbm9ybWFsIGRhdGEKICAgICAgICAgICAgIHggPSAxOjcsICAgICAgICAgICAgICAgICAgICAjIGZlYXR1cmVzIHRvIHVzZSB0byBnZW5lcmF0ZSB0aGUgcHJlZGljdGlvbgogICAgICAgICAgICAgeSA9IDksICAgICAgICAgICAgICAgICAgICAgICMgQ2xhc3MgdHlwZSAtPiB3aGF0IHdlIHdhbnQgdG8gcHJlZGljdAogICAgICAgICAgICAgbW9kZWxfaWQgPSAicmYxX3NSTkEiLCAgICAgICMgbmFtZSBvZiBtb2RlbCBpbiBoMm8KICAgICAgICAgICAgIG50cmVlcyA9IDQwMCwgICAgICAgICAgICAgICAjIG1heCBudW1iZXIgb2YgdHJlZXMgIAogICAgICAgICAgICAgc2VlZCA9IDEyMzQsICAgICAgICAgICAgICAgICMgc2VlZCwgaGFzIHRvIGJlIHNldCBXSVRISU4gdGhlIGgybyBmdW5jdGlvbiBhbmQgaXQncyBzdXBwb3NlZCB0byBiZSBkaWZmZXJlbnQgZnJvbSAiUidzIHNlZWQiLCBzbyByZXN1bHRzIG1pZ2h0IG5vdCBiZSBleGFjdGx5IHRoZSBzYW1lIGFzIG9yaWcgbW9kZWwsIGJ1dCBzaG91bGQgYmUgc2ltaWxhciBlbm91Z2gKICAgICAgICAgICAgIG10cmllcyA9IDIsICAgICAgICAgICAgICAgICAjIFNhbWUgYXMgb3JpZ2luYWwgbW9kZWwgCiAgICAgICAgICAgICBtYXhfZGVwdGggPSAzMAopCmBgYAoKIyMjIDYuMiBCdWlsZCBtb2RlbCB3aXRoIFNjYWxlZCBUcmFpbmluZyBkYXRhIApgYGB7cn0KcmZoMm9fc2NhbGVkIDwtIGgyby5yYW5kb21Gb3Jlc3QoIAogIHRyYWluaW5nX2ZyYW1lID0gdHJhaW5EYXRhX3NjYWxlZCwKICB4ID0gMTo3LCAgICAgICAgICAgICAgICAKICB5ID0gOSwgICAgICAgICAgICAgICAgICAgICAKICBtb2RlbF9pZCA9ICJyZjJfc1JOQSIsICAgICAKICBudHJlZXMgPSA0MDAsICAgICAgICAgICAgICAKICBzZWVkID0gMTIzNCwgICAgICAgICAgICAgIAogIG10cmllcyA9IDIsICAgICAgICAgICAgICAgIAogIG1heF9kZXB0aCA9IDMwCikKYGBgCgojIyMgNi4zIEJ1aWxkIG1vZGVsIHdpdGggTm9ybSBUcmFpbmluZyBkYXRhCmBgYHtyfQpyZmgyb19ub3JtIDwtIGgyby5yYW5kb21Gb3Jlc3QoIAogIHRyYWluaW5nX2ZyYW1lID0gdHJhaW5EYXRhX25vcm0sIAogIHggPSAxOjcsICAgICAgICAgICAgICAgICAgCiAgeSA9IDksICAgICAgICAgICAgICAgICAgICAKICBtb2RlbF9pZCA9ICJyZjNfc1JOQSIsICAgICAKICBudHJlZXMgPSA0MDAsICAgICAgICAgICAgICAgCiAgc2VlZCA9IDEyMzQsICAgICAgICAgICAgICAgCiAgbXRyaWVzID0gMiwgICAgICAgICAgICAgICAgICAKICBtYXhfZGVwdGggPSAzMAopCmBgYAoKIyMjIDYuNCBCT05VUzogQnVpbGQgR0xNIG1vZGVsCkFzIHdlIHdlcmUgZG9pbmcgdGVzdHMgd2l0aCBMSU1FLCB3ZSBkaXNjb3ZlcmVkIHRoYXQgaXQgeWllbGRlZCBpcnJlZ3VsYXIgcmVzcG9uc2VzLCBhbmQgYXMgYSByZXN1bHQsIHRoZSBleHBsYW5hdGlvbnMgY291bGQgbm90IGJlIHRydXN0ZWQgYXQgdGhlIHRpbWUuIEl0IHNlZW1zIHRvIGJlIGEga25vd24gaXNzdWUgdGhhdCBMSU1FIGlzIG5vdCBwZXJmZWN0IGZvciBldmVyeSBtb2RlbCwgYW5kIHRoYXQgaXQgc3RydWdnbGVzIHdpdGggZGF0YSB0aGF0IGlzIGhpZ2hseSBkaXZlcnNlLiBJbiB0aGlzIHVzZSBjYXNlLCBvdXIgZGF0YSBjb250YWlucyBtaXhlZCBudW1lcmljYWwgYW5kIGNhdGVnb3JpY2FsIGZlYXR1cmVzLCBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBzY2FsZXMgYW5kIG1hZ25pdHVkZXMgaW4gb3VyIG51bWVyY2lhbCBmZWF0dXJlcywgYW5kIGFuIGluYmFsYW5jZWQgZGF0YSBzZXRzIHRoYXQgZmF2b3JzIHByZWRpY3Rpb25zIG9mIHNSTkFzIGJlaW5nIGZhbHNlLiBTaW5jZSBsaW1lIGNyZWF0ZXMgYSBsb2NhbCBpbnRlcnByZXRhYmxlIGxpbmVhciBtb2RlbCBmb3IgaXRzIGV4cGxhbmF0aW9ucywgdGhlIEdMTSBjb25zdHJ1Y3RlZCBoZXJlIHdhcyBzaW1wbHkgdG8gdmVyaWZ5IHRoZSBoeXBvdGhlc2lzIHRoYXQgYSBsaW5lYXIgbW9kZWwgd291bGQgYmUgdW5hYmxlIHRvIGV4cGxhaW4gdGhlIGRhdGEuIFRoZSAidW5iYWxhbmNlZC1uZXNzIiBzZWVtcyB0byBjcmVhdGUgbW9kZWxzIHdoZXJlIHRoZSBmZWF0dXJlcyB0ZW5kIGFsd2F5cyBzdXBwb3J0IGEgcmVzdWx0IG9mICJmYWxzZSIsIHdoaWxlIGNsYWltaW5nIHRoYXQgYWxsIHRoZSBmZWF0dXJlcyBhcmUgYWdhaW5zdCBhIHJlc3VsdCBvZiB0cnVlLCB3aGljaCB3aWxsIGJlIGV2aWNlbmNlZCBpbiBzZWN0aW9uIDE1LjEuCgpgYGB7cn0KZ2xtaDJvIDwtIGgyby5nbG0oICMgaHR0cHM6Ly9kb2NzLmgyby5haS9oMm8vbGF0ZXN0LXN0YWJsZS9oMm8tZG9jcy9kYXRhLXNjaWVuY2UvZ2xtLmh0bWwgCiAgdHJhaW5pbmdfZnJhbWUgPSB0cmFpbkRhdGEsICMgdHJhaW5pbmcgdXNpbmcgdGhlIG5vcm1hbCBkYXRhCiAgeCA9IDE6NywgICAgICAgICAgICAgICAgICAgIyBmZWF0dXJlcyB0byB1c2UgdG8gZ2VuZXJhdGUgdGhlIHByZWRpY3Rpb24KICB5ID0gOSwgICAgICAgICAgICAgICAgICAgICAjIENsYXNzIHR5cGUgLT4gd2hhdCB3ZSB3YW50IHRvIHByZWRpY3QKICBtb2RlbF9pZCA9ICJnbG1fc1JOQSIsICAgICAjIG5hbWUgb2YgbW9kZWwgaW4gaDJvCiAgc2VlZCA9IDEyMzQsCiAgZmFtaWx5ID0gImJpbm9taWFsIgopCgpnbG1oMm9fcGVyZiA8LSBoMm8ucGVyZm9ybWFuY2UoZ2xtaDJvKQpnbG1oMm9fcGVyZgpoMm8uY29lZihnbG1oMm8pCmgyby5jb2VmX25vcm0oZ2xtaDJvKQpnbG1oMm9AbW9kZWwkY29lZmZpY2llbnRzX3RhYmxlCgpoMm8ucjIoZ2xtaDJvKSAjIFJeMiB2YWx1ZSBpcyByZWFsbHkgbG93LCBhbmQgaGF2aW5nIGEgTWF4IFJlY2FsbCBiZWluZyByZWFsbHkgbG93LCBtZWFuaW5nIHRoYXQgTElNRSBpcyBhbHNvIHByZWRpY3RpbmcgdGhhdCBldmVyeXRoaW5nIHNob3VsZCBiZSBmYWxzZQpgYGAKCiMjIDcuIFByZXZpZXcgSDJPJ3MgUkYgClRoaXMgaXMgdGhlIHBlcmZvcm1hbmNlIHdpdGggdGhlIE9PQiBlcnJvciwgYmFzZWQgb24gdGhlIHRyYWluaW5nIHByb2Nlc3MgKGllIHRyYWluaW5nIGRhdGEpCgpgYGB7cn0KcmZoMm8KcmZoMm9AbW9kZWwkdmFyaWFibGVfaW1wb3J0YW5jZXMKaDJvLnZhcmltcF9wbG90KHJmaDJvKQpyZmgyb190cmFpbmluZ19wZWZvcm1hbmNlIDwtIGgyby5wZXJmb3JtYW5jZShyZmgybykKcmZoMm9fdHJhaW5pbmdfcGVmb3JtYW5jZQpgYGAKCgojIElJSSkgQ09NUEFSRSBNT0RFTCdTIFBSRURJQ1RJT05TCkluIHRoaXMgc2VjdGlvbiB3ZSdsbCBnZXQgdGhlIHByZWRpY3Rpb25zIGFsbCB0aGUgbW9kZWxzIGdlbmVyYXRlIG9uIHRoZSB0ZXN0aW5nIGRhdGEgTFUgYW5kIFNMVDIuIFRoZSBnb2FsIG9mIHRoaXMgc2VjdGlvbiBpcyBub3QgdG8gcHJvdmUgd2hpY2ggbW9kZWwgaXMgdGhlIGJlc3QsIGJ1dCB0byBwcm92ZSB0aGF0IHRoZSBtb2RlbHMgYXJlIHNpbWlsYXIgZW5vdWdoIHRvIHRoZSBwb2ludCB0aGF0IHRoZSBhbmFseXNpcyBvZiB0aGUgSDJPIG1vZGVsIHdpbGwgYmUgYSB2YWxpZCBhbmFsb2d5IG1vZGVsIGZvciB0aGUgb3JpZ2luYWwgUkYgbW9kZWwuIAoKIyMgOC4gR2V0IFRlc3RpbmcgRGF0YSBQcmVkaWN0aW9ucyBmcm9tIHRoZSBNb2RlbHMKYGBge3J9Cm9yaWdSRl9sdV9wcmVkIDwtIHByZWRpY3Qob3JpZ1JGLCBsdWRhdGFbLC04XSwgdHlwZSA9ICJwcm9iIikKb3JpZ1JGX3NsdDJfcHJlZCA8LSBwcmVkaWN0KG9yaWdSRiwgc2x0MmRhdGFbLC04XSwgdHlwZSA9ICJwcm9iIikKCm9yaWdSRl9zY2FsZWRfbHVfcHJlZCA8LSBwcmVkaWN0KG9yaWdSRl9zY2FsZWQsIGx1ZGF0YV9zY2FsZWRbLC04XSwgdHlwZSA9ICJwcm9iIikKb3JpZ1JGX3NjYWxlZF9zbHQyX3ByZWQgPC0gcHJlZGljdChvcmlnUkZfc2NhbGVkLCBzbHQyZGF0YV9zY2FsZWRbLC04XSwgdHlwZSA9ICJwcm9iIikKCm9yaWdSRl9ub3JtX2x1X3ByZWQgPC0gcHJlZGljdChvcmlnUkZfbm9ybSwgbHVkYXRhX25vcm1bLC04XSwgdHlwZSA9ICJwcm9iIikKb3JpZ1JGX25vcm1fc2x0Ml9wcmVkIDwtIHByZWRpY3Qob3JpZ1JGX25vcm0sIHNsdDJkYXRhX25vcm1bLC04XSwgdHlwZSA9ICJwcm9iIikKCnJmaDJvX3NsdDJfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSByZmgybywgbmV3ZGF0YSA9IGFzLmgybyhzbHQyZGF0YVssLThdKSApCnJmaDJvX2x1X3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8obHVkYXRhWywtOF0pICkKCnJmaDJvX3NjYWxlZF9zbHQyX3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm9fc2NhbGVkLCBuZXdkYXRhID0gYXMuaDJvKHNsdDJkYXRhX3NjYWxlZFssLThdKSApCnJmaDJvX3NjYWxlZF9sdV9wcmVkIDwtIGgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX3NjYWxlZCwgbmV3ZGF0YSA9IGFzLmgybyhsdWRhdGFfc2NhbGVkWywtOF0pICkKCnJmaDJvX25vcm1fc2x0Ml9wcmVkIDwtIGgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX25vcm0sIG5ld2RhdGEgPSBhcy5oMm8oc2x0MmRhdGFfbm9ybVssLThdKSApCnJmaDJvX25vcm1fbHVfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSByZmgyb19ub3JtLCBuZXdkYXRhID0gYXMuaDJvKGx1ZGF0YV9ub3JtWywtOF0pICkKCmdsbV9zbHQyX3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gZ2xtaDJvLCBuZXdkYXRhID0gYXMuaDJvKHNsdDJkYXRhWywtOF0pICkKZ2xtX2x1X3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gZ2xtaDJvLCBuZXdkYXRhID0gYXMuaDJvKGx1ZGF0YVssLThdKSApCmBgYAoKIyMgOS4gQ3JlYXRlIGEgY29tcGFyaXNvbiBmdW5jdGlvbiAKVGhpcyBmdW5jdGlvbiBpcyB0byBzaW1wbGlmeSB0aGUgY29tcGFyaXNvbiBiZXR3ZWVuIHRoZSBwcmVkaWN0aW9ucyBmcm9tIHRoZSBPcmlnaW5hbCBSRiBtb2RlbCBhbmQgdGhlIEgyTyBSRiBtb2RlbC4gTGF0ZXIgb24gdGhpcywgdGhpcyBmdW5jdGlvbiB3aWxsIGJlIHVzZWQgdG8gY29tcGFyZSB0aGUgcHJlZGljdGlvbnMgYmV0d2VlbiB0aGUgT3JpZ1JGIG1vZGVsIGFuZCB0aGUgb3RoZXIgbW9kZWxzLiAKCmBgYHtyfQojIFRoaXMgZnVuY3Rpb24gd2lsbCBiZSB1c2VkIGxhdGVyIG9uLCBhbmQgCiMgICBhID0gdGhlIG9yaWdpbmFsIG1vZGVsJ3MgcHJlZGljdGlvbnMKIyAgIGIgPSB0aGUgYWx0ZXJuYXRlIG1vZGVsJ3MgcHJlZGljdGlvbnMKIyAgIGMgPSB0aGUgY29ycmVjdCBwcmVkaWN0aW9uCgpjb21wYXJlQW5zd2VycyA8LSBmdW5jdGlvbihhLGIsYyl7CiAgaWYgKCBhID09IGMgJiBiID09IGMgKSAKICAgICAgeyByZXM9IkJPVEhfUklHSFQiIH0KICBlbHNlIGlmICggYSA9PSBjICYgYiAhPSBjICkgCiAgICAgIHsgcmVzPSJPbmx5QSIgIH0KICBlbHNlIGlmICggYSAhPSBjICYgYiA9PSBjICkgCiAgICAgIHsgcmVzPSJPbmx5QiIgIH0KICBlbHNlIAogICAgICB7IHJlcz0iQk9USF9XUk9ORyIgIH0KICByZXR1cm4ocmVzKQp9CmBgYAoKIyMgMTAuIFNMVDIgY29tcGFyaXNvbnMgCiMjIyAxMC4xIEJ1aWxkIFNMVDIgUHJlZGljdGlvbnMgdGFibGUgCgpgYGB7cn0Kc2x0Ml9wcmVkaWN0aW9ucyA8LSAgY2JpbmQoYXMuZGF0YS5mcmFtZShzbHQyZGF0YSksICAgICAgICAgICAgICAgIyBJbnB1dAogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKG9yaWdSRl9zbHQyX3ByZWRbLDJdKSwgICAjIE9yaWcgUkYgcHJlZGljdGlvbnMgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUob3JpZ1JGX3NjYWxlZF9zbHQyX3ByZWRbLDJdKSwgICAjIE9yaWcgUkYgU2NhbGVkIHByZWRpY3Rpb25zIAogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKG9yaWdSRl9ub3JtX3NsdDJfcHJlZFssMl0pLCAgICMgT3JpZyBSRiBOb3JtYWxpemVkIHByZWRpY3Rpb25zICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUocmZoMm9fc2x0Ml9wcmVkWywzXSksICAgICAgICAjIEgyTyBSRiBwcmVkaWN0aW9ucwogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKHJmaDJvX3NjYWxlZF9zbHQyX3ByZWRbLDNdKSwgIyBTY2FsZWQgUkYgcHJlZGljdGlvbnMKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShyZmgyb19ub3JtX3NsdDJfcHJlZFssM10pICAgICMgTm9ybWFsaXplZCBSRiBwcmVkaWN0aW9ucwogICAgICAgICAgICAgICAgICAgICAgICAgICApCgpjb2xuYW1lcyhzbHQyX3ByZWRpY3Rpb25zKSA8LSBjKCJTUyIsICJQb3MxMHdydHNSTkFTdGFydCIsICJEaXN0VGVybSIsICJEaXN0YW5jZSIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJzYW1lU3RyYW5kIiwgIkRvd25EaXN0YW5jZSIsICJzYW1lRG93blN0cmFuZCIsICJDbGFzcyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIk9yaWdSRl9QIiwiT3JpZ1JGX3NjYWxlZF9QIiwiT3JpZ1JGX25vcm1fUCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIlJGX0gyT19QIiwiUkZIMk9fc2NhbGVkX1AiLCJSRkgyT19ub3JtX1AiKQoKc2x0Ml9wcmVkaWN0aW9ucyRvcmlnUHJlZHMgPC0gaWZlbHNlKCBzbHQyX3ByZWRpY3Rpb25zJE9yaWdSRl9QID49IDAuNSwgMSwgMCkgCnNsdDJfcHJlZGljdGlvbnMkb3JpZ1NjYWxlZFByZWRzIDwtIGlmZWxzZSggc2x0Ml9wcmVkaWN0aW9ucyRPcmlnUkZfc2NhbGVkX1AgPj0gMC41LCAxLCAwKSAKc2x0Ml9wcmVkaWN0aW9ucyRvcmlnTm9ybVByZWRzIDwtIGlmZWxzZSggc2x0Ml9wcmVkaWN0aW9ucyRPcmlnUkZfbm9ybV9QID49IDAuNSwgMSwgMCkgCnNsdDJfcHJlZGljdGlvbnMkaDJvUHJlZHMgIDwtIGlmZWxzZSggc2x0Ml9wcmVkaWN0aW9ucyRSRl9IMk9fUCA+PSAwLjUsIDEsIDApIApzbHQyX3ByZWRpY3Rpb25zJGgyb1NjYWxlZFByZWRzICA8LSBpZmVsc2UoIHNsdDJfcHJlZGljdGlvbnMkUkZIMk9fc2NhbGVkX1AgPj0gMC41LCAxLCAwKSAKc2x0Ml9wcmVkaWN0aW9ucyRoMm9Ob3JtUHJlZHMgIDwtIGlmZWxzZSggc2x0Ml9wcmVkaWN0aW9ucyRSRkgyT19ub3JtX1AgPj0gMC41LCAxLCAwKSAgCgpzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA8LSBOQQpzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPC0gTkEKc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA8LSBOQQpzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA8LSBOQQpzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPC0gTkEKCmZvciggaSBpbiAxOm5yb3coc2x0Ml9wcmVkaWN0aW9ucykgKXsKICAjIEJhc2VkIG9uIHRoZSB3YXkgd2UgYXJlIGZlZWRpbmcgdGhlIHZhbHVlcyB0byB0aGUgY29tcGFyZUFuc3dlcnMgZnVuY3Rpb24sIAogICMgU2ltaWxpYXJpdGllcyA9IE9ubHlBIG1lYW5zIHRoYXQgb25seSB0aGUgT3JpZyBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICMgU2ltaWxpYXJpdGllcyA9IE9ubHlCIG1lYW5zIHRoYXQgb25seSB0aGUgb3RoZXIgUkYgZ290IHRoZSByaWdodCBhbnN3ZXIKICAjIEFsbCBvdGhlciBhbnN3ZXJzIHdpbGwgdGVsbCB1cyB3aGVyZSBib3RoIG1vZGVscyB3ZXJlIHJpZ2h0KCJCT1RIX1JJR0hUIiksIG9yCiAgIyB3cm9uZyAoIkJPVEhfV1JPTkciKQogIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdWc09yaWdTY2FsZWQgPC0gY29tcGFyZUFuc3dlcnMoCiAgICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdTY2FsZWRQcmVkcyxzbHQyX3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzT3JpZ05vcm0gPC0gY29tcGFyZUFuc3dlcnMoCiAgICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdOb3JtUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnVnNIMk8gPC0gY29tcGFyZUFuc3dlcnMoCiAgICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJGgyb1ByZWRzLCBzbHQyX3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzSDJPU2NhbGVkIDwtIGNvbXBhcmVBbnN3ZXJzKAogICAgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ByZWRzLCBzbHQyX3ByZWRpY3Rpb25zW2ksXSRoMm9TY2FsZWRQcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQogIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdWc0gyT05vcm0gPC0gY29tcGFyZUFuc3dlcnMoCiAgICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJGgyb05vcm1QcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQp9IAoKcmJpbmQoIGhlYWQoc2x0Ml9wcmVkaWN0aW9ucywgNSksIHRhaWwoc2x0Ml9wcmVkaWN0aW9ucywgNSkgKQoKc3RyKHNsdDJfcHJlZGljdGlvbnMpCgpgYGAKCkFzIGFuIGFkZGl0aW9uYWwgY2hlY2ssIHdlIGFsc28gcmV2aWV3ZWQgdGhlIGNvcnJlbGF0aW9ucyBiZXR3ZWVuIHRoZSBPcmlnUkYgcHJlZGljdGlvbnMsIGFuZCB0aGUgYWx0ZXJuYXRpdmUgbW9kZWxzJ3MgcHJlZGljdGlvbnMgd2l0aCB0aGUgU0xUMiBwcmVkaWN0aW9ucy4gCgpgYGB7cn0KY29yKHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLHNsdDJfcHJlZGljdGlvbnNbLCJSRl9IMk9fUCJdKSAKY29yKHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfc2NhbGVkX1AiXSkKY29yKHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfbm9ybV9QIl0pCmNvcihzbHQyX3ByZWRpY3Rpb25zWywiT3JpZ1JGX1AiXSxzbHQyX3ByZWRpY3Rpb25zWywiUkZIMk9fc2NhbGVkX1AiXSkgCmNvcihzbHQyX3ByZWRpY3Rpb25zWywiT3JpZ1JGX1AiXSxzbHQyX3ByZWRpY3Rpb25zWywiUkZIMk9fbm9ybV9QIl0pCmBgYAoKIyMjIDEwLjIgQ29tcGFyZSBPcmlnUkYgdnMgdGhlIG90aGVyIE1vZGVscyB3aXRoIHRoZSBTTFQyIGRhdGEKSGVyZSwgd2UgYXJlIHNpbXBseSB0YWtpbmcgYSBsb29rIGF0IGhvdyBtYW55IGluc3RhbmNlcyB0aGUgT3JpZ1JGIG1vZGVsIGdvdCBjb3JyZWN0IHZzIHRoZSBvdGhlciBtb2RlbHMsIGFuZCBob3cgbWFueSBpbnN0YW5jZXMgYm90aCBtb2RlbHMgcHJlZGljdGVkIGNvcnJlY3RseSBvciB3cm9uZy4KICAqIEEgdmFsdWUgb2YgKk9ubHlBKiBtZWFucyB0aGF0IG9ubHkgdGhlIG9yaWcgUkYgZ290IHRoZSByaWdodCBhbnN3ZXIKICAqIEEgdmFsdWUgb2YgKk9ubHlCKiBtZWFucyB0aGF0IG9ubHkgdGhlIG90aGVyIFJGIGdvdCB0aGUgcmlnaHQgYW5zd2VyCiAgKiBBbGwgb3RoZXIgYW5zd2VycyB3aWxsIHRlbGwgdXMgd2hlcmUgYm90aCBtb2RlbHMgd2VyZSByaWdodCAqKCJCT1RIX1JJR0hUIikqIG9yIHdyb25nICooIkJPVEhfV1JPTkciKSoKCiMjIyMgMTAuMi4yIENvbXBhcmUgT3JpZ1JGIHZzIE9yaWdSRiBTY2FsZWQgTW9kZWwgCmBgYHtyfQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUEiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUIiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiQk9USF9XUk9ORyIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDEwLjIuMyBDb21wYXJlIE9yaWdSRiB2cyBPcmlnUkYgTm9ybSBNb2RlbCAKYGBge3J9Cm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiT25seUEiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIk9ubHlCIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJCT1RIX1dST05HIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDEwLjIuNCBDb21wYXJlIE9yaWdSRiB2cyBIMk8gTW9kZWwgCmBgYHtyfQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJPbmx5QSIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlCIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9XUk9ORyIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTAuMi41IENvbXBhcmUgT3JpZ1JGIHZzIEgyTyBNb2RlbCBTY2FsZWQgCgpgYGB7cn0KbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiT25seUEiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJPbmx5QiIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDEwLjIuNiBDb21wYXJlIE9yaWdSRiB2cyBIMk8gTW9kZWwgTm9ybWFsaXplZCAKYGBge3J9Cm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJPbmx5QSIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJPbmx5QiIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJCT1RIX1dST05HIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIDExLiBMVSBDb21wYXJpc29ucyAKIyMjIDExLjEgQnVpbGQgTFUgUHJlZGljdGlvbnMgdGFibGUgCgpgYGB7cn0KbHVfcHJlZGljdGlvbnMgPC0gIGNiaW5kKGFzLmRhdGEuZnJhbWUobHVkYXRhKSwgICAgICAgICAgICAgICAjIElucHV0CiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUob3JpZ1JGX2x1X3ByZWRbLDJdKSwgICAjIE9yaWcgUkYgcHJlZGljdGlvbnMgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUob3JpZ1JGX3NjYWxlZF9sdV9wcmVkWywyXSksICAgIyBPcmlnIFJGIFNjYWxlZCBwcmVkaWN0aW9ucyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShvcmlnUkZfbm9ybV9sdV9wcmVkWywyXSksICAgIyBPcmlnIFJGIE5vcm1hbGl6ZWQgcHJlZGljdGlvbnMgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShyZmgyb19sdV9wcmVkWywzXSksICAgICAgICAjIEgyTyBSRiBwcmVkaWN0aW9ucwogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKHJmaDJvX3NjYWxlZF9sdV9wcmVkWywzXSksICMgU2NhbGVkIFJGIHByZWRpY3Rpb25zCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUocmZoMm9fbm9ybV9sdV9wcmVkWywzXSkgICAgIyBOb3JtYWxpemVkIFJGIHByZWRpY3Rpb25zCikKCmNvbG5hbWVzKGx1X3ByZWRpY3Rpb25zKSA8LSBjKCJTUyIsICJQb3MxMHdydHNSTkFTdGFydCIsICJEaXN0VGVybSIsICJEaXN0YW5jZSIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJzYW1lU3RyYW5kIiwgIkRvd25EaXN0YW5jZSIsICJzYW1lRG93blN0cmFuZCIsICJDbGFzcyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIk9yaWdSRl9QIiwiT3JpZ1JGX3NjYWxlZF9QIiwiT3JpZ1JGX25vcm1fUCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIlJGX0gyT19QIiwiUkZIMk9fc2NhbGVkX1AiLCJSRkgyT19ub3JtX1AiKQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIApsdV9wcmVkaWN0aW9ucyRvcmlnUHJlZHMgPC0gaWZlbHNlKCBsdV9wcmVkaWN0aW9ucyRPcmlnUkZfUCA+PSAwLjUsIDEsIDApIApsdV9wcmVkaWN0aW9ucyRvcmlnU2NhbGVkUHJlZHMgPC0gaWZlbHNlKCBsdV9wcmVkaWN0aW9ucyRPcmlnUkZfc2NhbGVkX1AgPj0gMC41LCAxLCAwKSAKbHVfcHJlZGljdGlvbnMkb3JpZ05vcm1QcmVkcyA8LSBpZmVsc2UoIGx1X3ByZWRpY3Rpb25zJE9yaWdSRl9ub3JtX1AgPj0gMC41LCAxLCAwKSAKbHVfcHJlZGljdGlvbnMkaDJvUHJlZHMgIDwtIGlmZWxzZSggbHVfcHJlZGljdGlvbnMkUkZfSDJPX1AgPj0gMC41LCAxLCAwKSAKbHVfcHJlZGljdGlvbnMkaDJvU2NhbGVkUHJlZHMgIDwtIGlmZWxzZSggbHVfcHJlZGljdGlvbnMkUkZIMk9fc2NhbGVkX1AgPj0gMC41LCAxLCAwKSAKbHVfcHJlZGljdGlvbnMkaDJvTm9ybVByZWRzICA8LSBpZmVsc2UoIGx1X3ByZWRpY3Rpb25zJFJGSDJPX25vcm1fUCA+PSAwLjUsIDEsIDApICAKCmx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA8LSBOQQpsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkIDwtIE5BCmx1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtIDwtIE5BCmx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA8LSBOQQpsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtIDwtIE5BCgpmb3IoIGkgaW4gMTpucm93KGx1X3ByZWRpY3Rpb25zKSApewogICMgQmFzZWQgb24gdGhlIHdheSB3ZSBhcmUgZmVlZGluZyB0aGUgdmFsdWVzIHRvIHRoZSBjb21wYXJlQW5zd2VycyBmdW5jdGlvbiwgCiAgIyBTaW1pbGlhcml0aWVzID0gT25seUEgbWVhbnMgdGhhdCBvbmx5IHRoZSBPcmlnIFJGIGdvdCB0aGUgcmlnaHQgYW5zd2VyCiAgIyBTaW1pbGlhcml0aWVzID0gT25seUIgbWVhbnMgdGhhdCBvbmx5IHRoZSBvdGhlciBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICMgQWxsIG90aGVyIGFuc3dlcnMgd2lsbCB0ZWxsIHVzIHdoZXJlIGJvdGggbW9kZWxzIHdlcmUgcmlnaHQoIkJPVEhfUklHSFQiKSwgb3IKICAjIHdyb25nICgiQk9USF9XUk9ORyIpCiAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdWc09yaWdTY2FsZWQgPC0gY29tcGFyZUFuc3dlcnMoCiAgICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ByZWRzLCBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1NjYWxlZFByZWRzLGx1X3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdWc09yaWdOb3JtIDwtIGNvbXBhcmVBbnN3ZXJzKAogICAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdOb3JtUHJlZHMsICBsdV9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQogIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnVnNIMk8gPC0gY29tcGFyZUFuc3dlcnMoCiAgICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ByZWRzLCBsdV9wcmVkaWN0aW9uc1tpLF0kaDJvUHJlZHMsICAgICAgIGx1X3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdWc0gyT1NjYWxlZCA8LSBjb21wYXJlQW5zd2VycygKICAgIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIGx1X3ByZWRpY3Rpb25zW2ksXSRoMm9TY2FsZWRQcmVkcywgbHVfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzSDJPTm9ybSA8LSBjb21wYXJlQW5zd2VycygKICAgIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIGx1X3ByZWRpY3Rpb25zW2ksXSRoMm9Ob3JtUHJlZHMsICAgbHVfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKfSAKCnN0cihsdV9wcmVkaWN0aW9ucykKCnJiaW5kKCBoZWFkKGx1X3ByZWRpY3Rpb25zLCA1KSwgdGFpbChsdV9wcmVkaWN0aW9ucywgNSkpCgpgYGAKCkFzIGFuIGFkZGl0aW9uYWwgY2hlY2ssIHdlIGFsc28gcmV2aWV3IHRoZSBjb3JyZWxhdGlvbnMgYmV0d2VlbiB0aGUgT3JpZ1JGIHByZWRpY3Rpb25zLCBhbmQgdGhlIGFsdGVybmF0aXZlIG1vZGVscycgcHJlZGljdGlvbnMgd2l0aCB0aGUgTFUgZGF0YS4KCmBgYHtyfQpjb3IobHVfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLGx1X3ByZWRpY3Rpb25zWywiUkZfSDJPX1AiXSkgCmNvcihsdV9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sbHVfcHJlZGljdGlvbnNbLCJPcmlnUkZfc2NhbGVkX1AiXSkKY29yKGx1X3ByZWRpY3Rpb25zWywiT3JpZ1JGX1AiXSxsdV9wcmVkaWN0aW9uc1ssIk9yaWdSRl9ub3JtX1AiXSkKY29yKGx1X3ByZWRpY3Rpb25zWywiT3JpZ1JGX1AiXSxsdV9wcmVkaWN0aW9uc1ssIlJGSDJPX3NjYWxlZF9QIl0pIApjb3IobHVfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLGx1X3ByZWRpY3Rpb25zWywiUkZIMk9fbm9ybV9QIl0pCmBgYAoKCiMjIyAxMS4yIENvbXBhcmUgT3JpZ1JGIHZzIHRoZSBvdGhlciBNb2RlbHMgd2l0aCB0aGUgTFUgZGF0YQpCYXNpY2FsbHksIHRoZSBzYW1lIGV4cGxhbmF0aW9uIGFzIGluIHNlY3Rpb24gMTAuMgogICogQSB2YWx1ZSBvZiAqT25seUEqIG1lYW5zIHRoYXQgb25seSB0aGUgb3JpZyBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICogQSB2YWx1ZSBvZiAqT25seUIqIG1lYW5zIHRoYXQgb25seSB0aGUgb3RoZXIgUkYgZ290IHRoZSByaWdodCBhbnN3ZXIKICAqIEFsbCBvdGhlciBhbnN3ZXJzIHdpbGwgdGVsbCB1cyB3aGVyZSBib3RoIG1vZGVscyB3ZXJlIHJpZ2h0ICooIkJPVEhfUklHSFQiKSogb3Igd3JvbmcgKigiQk9USF9XUk9ORyIpKgoKIyMjIyAxMS4yLjEgQ29tcGFyZSBPcmlnUkYgdnMgT3JpZ1JGIFNjYWxlZCBNb2RlbCAKYGBge3J9Cm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIk9ubHlBIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUIiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1dST05HIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiQk9USF9SSUdIVCIsXSApCmBgYAoKIyMjIyAxMS4yLjIgQ29tcGFyZSBPcmlnUkYgdnMgT3JpZ1JGIE5vcm0gTW9kZWwgCmBgYHtyfQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiT25seUEiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiT25seUIiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiQk9USF9XUk9ORyIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDExLjIuMyBDb21wYXJlIE9yaWdSRiB2cyBIMk8gTW9kZWwgCmBgYHtyfQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlBIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJPbmx5QiIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9XUk9ORyIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9SSUdIVCIsXSApCmBgYAoKIyMjIyAxMS4yLjQgQ29tcGFyZSBPcmlnUkYgdnMgSDJPIE1vZGVsIFNjYWxlZCAKYGBge3J9Cm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiT25seUEiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIk9ubHlCIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJCT1RIX1dST05HIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDExLjIuNSBDb21wYXJlIE9yaWdSRiB2cyBIMk8gTW9kZWwgTm9ybWFsaXplZCAKYGBge3J9Cm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIk9ubHlBIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiT25seUIiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJCT1RIX1dST05HIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiQk9USF9SSUdIVCIsXSApCmBgYAoKIyBJVikgQ09NUEFSRSBNT0RFTCdTIE1FVFJJQ1MgCgpXaGlsZSBzZWN0aW9ucyAxMS4yIGFuZCAxMC4yIHN1Z2dlc3QgdGhhdCBhbGwgdGhlIG1vZGVscyBhcmUgYmVoYXZpbmcgc2ltaWxhcmx5LCBtYWtpbmcgYSBqdWRnZW1lbnQgc29sZWx5IG9uIGFjY3VyYWN5IGNhbiBiZSBkZWNlaXZpbmcuIEZvciB0aGlzIHJlYXNvbiwgd2UgYWxzbyBkZWNpZGVkIHRvIGRpdmUgZGVlcGVyIGludG8gdGhlIG1vZGVsJ3MgcGVyZm9ybWFuY2VzLCBhbmQgZG8gYWRkaXRpb25hbCBzaWRlIGJ5IHNpZGUgY29tcGFyaXNvbiBvZiBvdGhlciBwZXJmb3JtYW5jZSBtZXRyaWNzLiAKCiMjIDEyLiBHZXQgUGVyZm9ybWFuY2UgTWV0cmljcyAKVGhpcyAqZXZhbHVhdGVEYXRhKiBmdW5jdGlvbiB3YXMgY29waWVkIGRpcmVjdGx5IGZyb20gdGhlIHNvdXJjZSBtYXRlcmlhbHModGhlIFthZGRpdGlvbmFsIG1hdGVyaWFsXShodHRwczovL3BlZXJqLmNvbS9hcnRpY2xlcy82MzA0LykgaW5jbHVkZWQgaW4gdGhlIG9yaWdpbmFsIGFydGljbGUpLCBhcyBpdCB3YXMgdGhlIG9uZSB1c2VkIHRvIGV2YWx1YXRlIHRoZSBvcmlnaW5hbCBtb2RlbCwgYW5kIHdlJ3ZlIHNpbXBseSBhZGRlZCBjb21tZW50cyBhIGZldyBleHRyYSBjb21tZW50cy4gVGhlIEgyTyBsaWJyYXJ5IGFscmVhZHkgY29udGFpbnMgdG9vbHMgdG8gb2J0YWluIHBlcmZvcm1hbmNlIG1ldHJpY3MgZnJvbSBpdHMgbW9kZWxzLCBzbyBubyBhZGRpdGlvbmFsIGZ1bmN0aW9ucyB3ZXJlIG5lY2Vzc2FyeSB3aXRoIHRoZSBIMk8gTUwgbW9kZWxzLiAgCgpgYGB7cn0KZXZhbHVhdGVEYXRhIDwtIGZ1bmN0aW9uKFJGLCBkYXRhLCBsYWJlbHMpewogIHJlcXVpcmUoUk9DUikKICByZXF1aXJlKFBSUk9DKQogIHJlcXVpcmUocmFuZG9tRm9yZXN0KQogIHJlcyA8LSBsaXN0KCkKICByZXMkcHJlZEQgPC0gcHJlZGljdChSRiwgZGF0YSwgdHlwZSA9ICJwcm9iIikgICAgICAgICAgICAgICAgICAgICAgICAjIFByZWRpY3Rpb25zIGFzIEdlbmVyYXRlZCBieSB0aGUgbW9kZWwKICByZXMkcHJlZCA8LSBwcmVkaWN0aW9uKHJlcyRwcmVkRFssMl0sIGFzLmxvZ2ljYWwobGFiZWxzKSkgICAgICAgICAgICAjIFByZWRpY3Rpb25zLCBidXQgY29udmVydGVkIHRvIFM0IE9iamVjdCAgICAgICAgICAKICByZXMkUFIgPC0gcGVyZm9ybWFuY2UocmVzJHByZWQsIG1lYXN1cmUgPSAicHJlYyIsIHgubWVhc3VyZSA9ICJyZWMiKSAjIFByZWNpc2lvbiBSZWNhbGwgQ3VydmUKICByZXMkU1MgPC0gcGVyZm9ybWFuY2UocmVzJHByZWQsIG1lYXN1cmU9InNlbnMiLCB4Lm1lYXN1cmU9InNwZWMiKSAgICAjIFNlbnNpdGl2aXR5IHZzLiBTcGVjaWZpY2l0eQogIHJlcyRhdWMgIDwtIHBlcmZvcm1hbmNlKHJlcyRwcmVkLCBtZWFzdXJlID0gImF1YyIpICAgICAgICAgICAgICAgICAgICMgU2Vuc2l0aXZpdHkgdnMuIFNwZWNpZmljaXR5IEFVQwogIHJlcyRhY2MgPC0gIHBlcmZvcm1hbmNlKHJlcyRwcmVkLCBtZWFzdXJlID0gImFjYyIpICAgICAgICAgICAgICAgICAgICMgQWNjdXJhY3kKICByZXMkcHIgPC0gcHIuY3VydmUoc2NvcmVzLmNsYXNzMCA9IHJlcyRwcmVkRFtsYWJlbHMgPT0gMSwyXSwgICAgICAgICAjIFByZWNpc2lvbiB2cy4gUmVjYWxsIAogICAgICAgICAgICAgICAgICAgICBzY29yZXMuY2xhc3MxID0gcmVzJHByZWREW2xhYmVscyA9PSAwLDJdLCAKICAgICAgICAgICAgICAgICAgICAgY3VydmUgPSBUKQogIHJldHVybihyZXMpCn0KCiMgT3JpZ1JGIFBlcmZvcm1hbmNlIE1ldHJpY3MKb3JpZ1JGX3NsdDJfcGVyZm9ybWFuY2UgPC0gZXZhbHVhdGVEYXRhKG9yaWdSRiwgc2x0MmRhdGFbLC04XSwgc2x0MmRhdGFbLDhdKQpvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UgPC0gZXZhbHVhdGVEYXRhKG9yaWdSRl9zY2FsZWQsIHNsdDJkYXRhX3NjYWxlZFssLThdLCBzbHQyZGF0YV9zY2FsZWRbLDhdKQpvcmlnUkZfbm9ybV9zbHQyX3BlcmZvcm1hbmNlIDwtIGV2YWx1YXRlRGF0YShvcmlnUkZfbm9ybSwgc2x0MmRhdGFfbm9ybVssLThdLCBzbHQyZGF0YV9ub3JtWyw4XSkKCm9yaWdSRl9sdV9wZXJmb3JtYW5jZSA8LSBldmFsdWF0ZURhdGEob3JpZ1JGLCBsdWRhdGFbLC04XSwgbHVkYXRhWyw4XSkKb3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSA8LSBldmFsdWF0ZURhdGEob3JpZ1JGX3NjYWxlZCwgbHVkYXRhX3NjYWxlZFssLThdLCBsdWRhdGFfc2NhbGVkWyw4XSkKb3JpZ1JGX25vcm1fbHVfcGVyZm9ybWFuY2UgPC0gZXZhbHVhdGVEYXRhKG9yaWdSRl9ub3JtLCBsdWRhdGFfbm9ybVssLThdLCBsdWRhdGFfbm9ybVssOF0pCgoKIyBIMk8gUkYgUGVyZm9ybWFuY2UgTWV0cmljcwojIEgyTyBwcm92aWRlcyBhIHdheSB0byByZXRyaWV2ZSBtb3N0IG9mIHRoZSBkZXNpcmVkIG1ldHJpY3MKIyBodHRwOi8vZG9jcy5oMm8uYWkvaDJvL2xhdGVzdC1zdGFibGUvaDJvLXIvZG9jcy9yZWZlcmVuY2UvaDJvLm1ldHJpYy5odG1sCgpzbHQyZGF0YV9oMm8gPC0gc2x0MmRhdGEKc2x0MmRhdGFfaDJvWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKHNsdDJkYXRhX2gyb1ssIkNsYXNzIl0pIApyZmgyb19zbHQyX3BlcmZvcm1hbmNlIDwtIGgyby5wZXJmb3JtYW5jZShyZmgybywgbmV3ZGF0YSA9IGFzLmgybyhzbHQyZGF0YV9oMm8pKQoKbHVkYXRhX2gybyA8LSBsdWRhdGEKbHVkYXRhX2gyb1ssIkNsYXNzIl0gPC0gYXMubG9naWNhbChsdWRhdGFfaDJvWywiQ2xhc3MiXSkgCnJmaDJvX2x1X3BlcmZvcm1hbmNlIDwtIGgyby5wZXJmb3JtYW5jZShyZmgybywgbmV3ZGF0YSA9IGFzLmgybyhsdWRhdGFfaDJvKSkKCnNsdDJkYXRhX2gyb19zY2FsZWQgPC0gc2x0MmRhdGFfc2NhbGVkCnNsdDJkYXRhX2gyb19zY2FsZWRbLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwoc2x0MmRhdGFfaDJvX3NjYWxlZFssIkNsYXNzIl0pIApyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZCA8LSBoMm8ucGVyZm9ybWFuY2UocmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8oc2x0MmRhdGFfaDJvX3NjYWxlZCkpCgpsdWRhdGFfaDJvX3NjYWxlZCA8LSBsdWRhdGFfc2NhbGVkCmx1ZGF0YV9oMm9fc2NhbGVkWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKGx1ZGF0YV9oMm9fc2NhbGVkWywiQ2xhc3MiXSkgCnJmaDJvX2x1X3BlcmZvcm1hbmNlX3NjYWxlZCA8LSBoMm8ucGVyZm9ybWFuY2UocmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8obHVkYXRhX2gyb19zY2FsZWQpKQoKc2x0MmRhdGFfaDJvX25vcm0gPC0gc2x0MmRhdGFfbm9ybQpzbHQyZGF0YV9oMm9fbm9ybVssIkNsYXNzIl0gPC0gYXMubG9naWNhbChzbHQyZGF0YV9oMm9fbm9ybVssIkNsYXNzIl0pIApyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0gPC0gaDJvLnBlcmZvcm1hbmNlKHJmaDJvLCBuZXdkYXRhID0gYXMuaDJvKHNsdDJkYXRhX2gyb19ub3JtKSkKCmx1ZGF0YV9oMm9fbm9ybSA8LSBsdWRhdGFfbm9ybQpsdWRhdGFfaDJvX25vcm1bLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwobHVkYXRhX2gyb19ub3JtWywiQ2xhc3MiXSkgCnJmaDJvX2x1X3BlcmZvcm1hbmNlX25vcm0gPC0gaDJvLnBlcmZvcm1hbmNlKHJmaDJvLCBuZXdkYXRhID0gYXMuaDJvKGx1ZGF0YV9oMm9fbm9ybSkpCmBgYAoKIyMgMTMuIENvbXBhcmUgTWV0cmljcyAgCiMjIyAxMy4xIEFjY3VyYWN5CiMjIyMgMTMuMS4xIEFjY3VyYWN5IFRhYmxlCmBgYHtyfQptZXRyaWNzX3RhYmxlIDwtIGRhdGEuZnJhbWUoIkFjY3VyYWN5IiA9IDE6MTIpCnJvd25hbWVzKG1ldHJpY3NfdGFibGUpIDwtIGMoIm9yaWdSRl9zbHQyX3BlcmYiLCAib3JpZ1JGX3NjYWxlZF9zbHQyX3BlcmYiLCAib3JpZ1JGX25vcm1fc2x0Ml9wZXJmIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAicmZoMm9fc2x0Ml9wZXJmIiwgInJmaDJvX3NsdDJfcGVyZl9zY2FsZWQiLCAicmZoMm9fc2x0Ml9wZXJmX25vcm0iLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICJvcmlnUkZfbHVfcGVyZiIsICJvcmlnUkZfc2NhbGVkX2x1X3BlcmYiLCAib3JpZ1JGX25vcm1fbHVfcGVyZiIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICJyZmgyb19sdV9wZXJmIiwgICJyZmgyb19sdV9wZXJmX3NjYWxlZCIsICAicmZoMm9fbHVfcGVyZl9ub3JtIikKCm1ldHJpY3NfdGFibGVbIm9yaWdSRl9zbHQyX3BlcmYiLCJBY2N1cmFjeSJdICAgICAgICA8LSBucm93KHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1JJR0hUIiB8IHNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUEiLF0gKS8KICBucm93KHNsdDJfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbIm9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmIiwiQWNjdXJhY3kiXSA8LSBucm93KHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1JJR0hUIiB8IHNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUIiLF0gKS8KICBucm93KHNsdDJfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbIm9yaWdSRl9ub3JtX3NsdDJfcGVyZiIsIkFjY3VyYWN5Il0gICA8LSBucm93KHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiQk9USF9SSUdIVCIgfCBzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJPbmx5QiIsXSApLwogIG5yb3coc2x0Ml9wcmVkaWN0aW9ucykKCm1ldHJpY3NfdGFibGVbInJmaDJvX3NsdDJfcGVyZiIsIkFjY3VyYWN5Il0gICAgICAgICA8LSBucm93KHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIkJPVEhfUklHSFQiIHwgc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlCIixdICkvCiAgbnJvdyhzbHQyX3ByZWRpY3Rpb25zKQptZXRyaWNzX3RhYmxlWyJyZmgyb19zbHQyX3BlcmZfc2NhbGVkIiwiQWNjdXJhY3kiXSAgPC0gbnJvdyhzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJCT1RIX1JJR0hUIiB8ICBzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiT25seUIiLF0pLwogIG5yb3coc2x0Ml9wcmVkaWN0aW9ucykKbWV0cmljc190YWJsZVsicmZoMm9fc2x0Ml9wZXJmX25vcm0iLCJBY2N1cmFjeSJdICAgIDwtIG5yb3coc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIkJPVEhfUklHSFQiIHwgc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJPbmx5QiIsXSkvCiAgbnJvdyhzbHQyX3ByZWRpY3Rpb25zKQoKbWV0cmljc190YWJsZVsib3JpZ1JGX2x1X3BlcmYiLCJBY2N1cmFjeSJdICAgICAgICAgIDwtIG5yb3cobHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJCT1RIX1JJR0hUIiB8IGx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiT25seUEiLF0gKS8KICBucm93KGx1X3ByZWRpY3Rpb25zKQptZXRyaWNzX3RhYmxlWyJvcmlnUkZfc2NhbGVkX2x1X3BlcmYiLCJBY2N1cmFjeSJdICAgPC0gbnJvdyhsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1JJR0hUIiB8IGx1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIk9ubHlCIixdICkvCiAgbnJvdyhsdV9wcmVkaWN0aW9ucykKbWV0cmljc190YWJsZVsib3JpZ1JGX25vcm1fbHVfcGVyZiIsIkFjY3VyYWN5Il0gICAgIDwtIG5yb3cobHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIkJPVEhfUklHSFQiIHwgbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIk9ubHlCIixdICkvCiAgbnJvdyhsdV9wcmVkaWN0aW9ucykKCm1ldHJpY3NfdGFibGVbInJmaDJvX2x1X3BlcmYiLCJBY2N1cmFjeSJdICAgICAgICAgICA8LSBucm93KGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9SSUdIVCIgfCAgbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJPbmx5QiIsXSApLwogIG5yb3cobHVfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbInJmaDJvX2x1X3BlcmZfc2NhbGVkIiwiQWNjdXJhY3kiXSAgICA8LSBucm93KGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiQk9USF9SSUdIVCIgfCBsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIk9ubHlCIixdKS8KICBucm93KGx1X3ByZWRpY3Rpb25zKQptZXRyaWNzX3RhYmxlWyJyZmgyb19sdV9wZXJmX25vcm0iLCJBY2N1cmFjeSJdICAgICAgPC0gbnJvdyhsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJCT1RIX1JJR0hUIiB8IGx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIk9ubHlCIixdICkvCiAgbnJvdyhsdV9wcmVkaWN0aW9ucykKCm1ldHJpY3NfdGFibGUKYGBgCgojIyMjIDEzLjEuMiBBY2N1cmFjeSBncmFwaHMgYmFzZWQgb24gZGlmZmVyZW50IHRocmVzaG9sZHMoImN1dG9mZiIpCgpUaGUgTGVnZW5kIGZvciB0aGUgZ3JhcGhzOgogICogIEJsdWUgPSBPcmlnUkYKICAqICBNYWdlbnRhID0gT3JpZ1JGIGJ1dCBzY2FsZWQKICAqICBCbGFjayA9IE9yaWdSRiBidXQgbm9ybWFsaXplZAogICogIFJlZCA9IEgyTyBSRiBtb2RlbAogICogIEJyb3duID0gSDJPIFJGIG1vZGVsIHdpdGggc2NhbGVkIGRhdGEKICAqICBHcmVlbiAgPSBIMk8gUkYgbW9kZWwgd2l0aCBub3JtYWxpemVkIGRhdGEKCmBgYHtyfQojIFNMVDIgQWNjdXJhY3kgUGxvdApwbG90KG9yaWdSRl9zbHQyX3BlcmZvcm1hbmNlJGFjYywgY29sID0gImJsdWUiLCBsd2QgPSA1LCBtYWluID0gIlNMVDIgQWNjdXJhY3kiKSAKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkYWNjQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX3NjYWxlZF9zbHQyX3BlcmZvcm1hbmNlJGFjY0B5LnZhbHVlcykpLCBjb2wgPSAibWFnZW50YSIsIGx3ZCA9IDQpIApsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX3NsdDJfcGVyZm9ybWFuY2UkYWNjQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRhY2NAeS52YWx1ZXMpKSwgY29sID0gImJsYWNrIiwgbHdkID0gMSkgCmxpbmVzKGgyby5hY2N1cmFjeShyZmgyb19zbHQyX3BlcmZvcm1hbmNlKSwgdHlwZSA9ICJsIiwgY29sID0gInJlZCIsIGx3ZCA9IDMpCmxpbmVzKGgyby5hY2N1cmFjeShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZCksIHR5cGUgPSAibCIsIGNvbCA9ICJicm93biIsIGx3ZCA9IDMpCmxpbmVzKGgyby5hY2N1cmFjeShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pLCB0eXBlID0gImwiLCBjb2wgPSAiZ3JlZW4iLCBsd2QgPSAzKQoKIyBMVSBBY2N1cmFjeSBQbG90CnBsb3Qob3JpZ1JGX2x1X3BlcmZvcm1hbmNlJGFjYywgY29sID0gImJsdWUiLCBsd2QgPSA1LCBtYWluID0gIkxVIEFjY3VyYWN5IikKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlJGFjY0B4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfbHVfcGVyZm9ybWFuY2UkYWNjQHkudmFsdWVzKSksIGNvbCA9ICJtYWdlbnRhIiwgbHdkID0gNCkgCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fbHVfcGVyZm9ybWFuY2UkYWNjQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fbHVfcGVyZm9ybWFuY2UkYWNjQHkudmFsdWVzKSksIGNvbCA9ICJibGFjayIsIGx3ZCA9IDEpIApsaW5lcyhoMm8uYWNjdXJhY3kocmZoMm9fbHVfcGVyZm9ybWFuY2UpLCB0eXBlID0gImwiLCBjb2wgPSAicmVkIiwgbHdkID0gMykKbGluZXMoaDJvLmFjY3VyYWN5KHJmaDJvX2x1X3BlcmZvcm1hbmNlX3NjYWxlZCksIHR5cGUgPSAibCIsIGNvbCA9ICJicm93biIsIGx3ZCA9IDMpCmxpbmVzKGgyby5hY2N1cmFjeShyZmgyb19sdV9wZXJmb3JtYW5jZV9ub3JtKSwgdHlwZSA9ICJsIiwgY29sID0gImdyZWVuIiwgbHdkID0gMykKYGBgCgojIyMgMTMuMiBBVUNQUiAKVGhlIExlZ2VuZCBmb3IgdGhlIGdyYXBoczoKICAqICBCbHVlID0gT3JpZ1JGCiAgKiAgTWFnZW50YSA9IE9yaWdSRiBidXQgc2NhbGVkCiAgKiAgQmxhY2sgPSBPcmlnUkYgYnV0IG5vcm1hbGl6ZWQKICAqICBSZWQgPSBIMk8gUkYgbW9kZWwKICAqICBCcm93biA9IEgyTyBSRiBtb2RlbCB3aXRoIHNjYWxlZCBkYXRhCiAgKiAgR3JlZW4gID0gSDJPIFJGIG1vZGVsIHdpdGggbm9ybWFsaXplZCBkYXRhCmBgYHtyfQptZXRyaWNzX3RhYmxlWyJvcmlnUkZfc2x0Ml9wZXJmIiwiQVVDUFIiXSA8LSBvcmlnUkZfc2x0Ml9wZXJmb3JtYW5jZSRwciRhdWMuaW50ZWdyYWwKbWV0cmljc190YWJsZVsib3JpZ1JGX3NjYWxlZF9zbHQyX3BlcmYiLCJBVUNQUiJdIDwtIG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRwciRhdWMuaW50ZWdyYWwKbWV0cmljc190YWJsZVsib3JpZ1JGX25vcm1fc2x0Ml9wZXJmIiwiQVVDUFIiXSA8LSBvcmlnUkZfbm9ybV9zbHQyX3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJyZmgyb19zbHQyX3BlcmYiLCAiQVVDUFIiXSA8LSBoMm8uYXVjcHIocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSkKbWV0cmljc190YWJsZVsicmZoMm9fc2x0Ml9wZXJmX3NjYWxlZCIsICJBVUNQUiJdIDwtIGgyby5hdWNwcihyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZCkKbWV0cmljc190YWJsZVsicmZoMm9fc2x0Ml9wZXJmX25vcm0iLCAiQVVDUFIiXSA8LSBoMm8uYXVjcHIocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9ub3JtKQoKbWV0cmljc190YWJsZVsib3JpZ1JGX2x1X3BlcmYiLCAgIkFVQ1BSIl0gPC0gb3JpZ1JGX2x1X3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJvcmlnUkZfc2NhbGVkX2x1X3BlcmYiLCAgIkFVQ1BSIl0gPC0gb3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSRwciRhdWMuaW50ZWdyYWwKbWV0cmljc190YWJsZVsib3JpZ1JGX25vcm1fbHVfcGVyZiIsICAiQVVDUFIiXSA8LSBvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRwciRhdWMuaW50ZWdyYWwKbWV0cmljc190YWJsZVsicmZoMm9fbHVfcGVyZiIsICAiQVVDUFIiXSA8LSBoMm8uYXVjcHIocmZoMm9fbHVfcGVyZm9ybWFuY2UpCm1ldHJpY3NfdGFibGVbInJmaDJvX2x1X3BlcmZfc2NhbGVkIiwgIkFVQ1BSIl0gPC0gaDJvLmF1Y3ByKHJmaDJvX2x1X3BlcmZvcm1hbmNlX3NjYWxlZCkKbWV0cmljc190YWJsZVsicmZoMm9fbHVfcGVyZl9ub3JtIiwgIkFVQ1BSIl0gPC0gaDJvLmF1Y3ByKHJmaDJvX2x1X3BlcmZvcm1hbmNlX25vcm0pCgptZXRyaWNzX3RhYmxlCgpwbG90KG9yaWdSRl9zbHQyX3BlcmZvcm1hbmNlJFBSLCAgY29sID0gImJsdWUiLCBsd2QgPSA1LCBtYWluID0gIlNMVDIgUFIgQ3VydmUiICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkUFJAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkUFJAeS52YWx1ZXMpKSwgIGNvbCA9ICJtYWdlbnRhIiwgbHdkID0gNCApCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRQUkB4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX3NsdDJfcGVyZm9ybWFuY2UkUFJAeS52YWx1ZXMpKSwgIGNvbCA9ICJibGFjayIsIGx3ZCA9IDIgKQoKbGluZXMoeCA9IGgyby5yZWNhbGwocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSlbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSlbLCJwcmVjaXNpb24iXSwKICAgICAgY29sID0gInJlZCIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQpsaW5lcyh4ID0gaDJvLnJlY2FsbChyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZClbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9zY2FsZWQpWywicHJlY2lzaW9uIl0sCiAgICAgIGNvbCA9ICJicm93biIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQpsaW5lcyh4ID0gaDJvLnJlY2FsbChyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pWywidHByIl0sCiAgICAgIHkgPSBoMm8ucHJlY2lzaW9uKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSlbLCJwcmVjaXNpb24iXSwKICAgICAgY29sID0gImdyZWVuIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCgoKcGxvdChvcmlnUkZfbHVfcGVyZm9ybWFuY2UkUFIsICBjb2wgPSAiYmx1ZSIsIGx3ZCA9IDUsIG1haW4gPSAiTFUgUFIgQ3VydmUiICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlJFBSQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSRQUkB5LnZhbHVlcykpLCAgY29sID0gIm1hZ2VudGEiLCBsd2QgPSA0ICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRQUkB4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX2x1X3BlcmZvcm1hbmNlJFBSQHkudmFsdWVzKSksICBjb2wgPSAiYmxhY2siLCBsd2QgPSAyICkKbGluZXMoeCA9IGgyby5yZWNhbGwocmZoMm9fbHVfcGVyZm9ybWFuY2UpWywidHByIl0sCiAgICAgIHkgPSBoMm8ucHJlY2lzaW9uKHJmaDJvX2x1X3BlcmZvcm1hbmNlKVssInByZWNpc2lvbiJdLAogICAgICBjb2wgPSAicmVkIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmxpbmVzKHggPSBoMm8ucmVjYWxsKHJmaDJvX2x1X3BlcmZvcm1hbmNlX3NjYWxlZClbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKVssInByZWNpc2lvbiJdLAogICAgICBjb2wgPSAiYnJvd24iLCB0eXBlID0gImwiLCBsd2QgPSAzCikKbGluZXMoeCA9IGgyby5yZWNhbGwocmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSlbLCJwcmVjaXNpb24iXSwKICAgICAgY29sID0gImdyZWVuIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmBgYAoKIyMjIDEzLjMgU2Vuc2l0aXZ5IHZzIFNwZWNpZmljaXR5IEdyYXBoIApUaGUgTGVnZW5kIGZvciB0aGUgZ3JhcGhzOgogICogIEJsdWUgPSBPcmlnUkYKICAqICBNYWdlbnRhID0gT3JpZ1JGIGJ1dCBzY2FsZWQKICAqICBCbGFjayA9IE9yaWdSRiBidXQgbm9ybWFsaXplZAogICogIFJlZCA9IEgyTyBSRiBtb2RlbAogICogIEJyb3duID0gSDJPIFJGIG1vZGVsIHdpdGggc2NhbGVkIGRhdGEKICAqICBHcmVlbiAgPSBIMk8gUkYgbW9kZWwgd2l0aCBub3JtYWxpemVkIGRhdGEKYGBge3J9CnBsb3Qob3JpZ1JGX3NsdDJfcGVyZm9ybWFuY2UkU1MsIGNvbCA9ICJibHVlIiwgbHdkID0gNSwgbWFpbiA9ICJTTFQyIFNlbnNpdGl2aXR5IHZzIFNwZWNpZmljaXR5IikKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkU1NAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkU1NAeS52YWx1ZXMpKSwgIGNvbCA9ICJtYWdlbnRhIiwgbHdkID0gNCApCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRTU0B4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX3NsdDJfcGVyZm9ybWFuY2UkU1NAeS52YWx1ZXMpKSwgIGNvbCA9ICJibGFjayIsIGx3ZCA9IDIgKQoKbGluZXMoeCA9IGgyby5zcGVjaWZpY2l0eShyZmgyb19zbHQyX3BlcmZvcm1hbmNlKVssInRuciJdLAogICAgIHkgPSBoMm8uc2Vuc2l0aXZpdHkocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSlbLCJ0cHIiXSwKICAgICBjb2wgPSAicmVkIiwgdHlwZSA9ICJsIiwgbHdkID0gMwogICAgICkKCmxpbmVzKHggPSBoMm8uc3BlY2lmaWNpdHkocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9zY2FsZWQpWywidG5yIl0sCiAgICAgIHkgPSBoMm8uc2Vuc2l0aXZpdHkocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9zY2FsZWQpWywidHByIl0sCiAgICAgIGNvbCA9ICJicm93biIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQpsaW5lcyh4ID0gaDJvLnNwZWNpZmljaXR5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0bnIiXSwKICAgICAgeSA9IGgyby5zZW5zaXRpdml0eShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pWywidHByIl0sCiAgICAgIGNvbCA9ICJncmVlbiIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQoKcGxvdChvcmlnUkZfbHVfcGVyZm9ybWFuY2UkU1MsIGNvbCA9ICJibHVlIiwgbHdkID0gNSwgbWFpbiA9ICJMVSBTZW5zaXRpdml0eSB2cyBTcGVjaWZpY2l0eSIpCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSRTU0B4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfbHVfcGVyZm9ybWFuY2UkU1NAeS52YWx1ZXMpKSwgIGNvbCA9ICJtYWdlbnRhIiwgbHdkID0gNCApCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fbHVfcGVyZm9ybWFuY2UkU1NAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRTU0B5LnZhbHVlcykpLCAgY29sID0gImJsYWNrIiwgbHdkID0gMiApCmxpbmVzKHggPSBoMm8uc3BlY2lmaWNpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2UpWywidG5yIl0sCiAgICAgIHkgPSBoMm8uc2Vuc2l0aXZpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2UpWywidHByIl0sCiAgICAgIGNvbCA9ICJyZWQiLCB0eXBlID0gImwiLCBsd2QgPSAzCikKCmxpbmVzKHggPSBoMm8uc3BlY2lmaWNpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKVssInRuciJdLAogICAgICB5ID0gaDJvLnNlbnNpdGl2aXR5KHJmaDJvX2x1X3BlcmZvcm1hbmNlX3NjYWxlZClbLCJ0cHIiXSwKICAgICAgY29sID0gImJyb3duIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCgpsaW5lcyh4ID0gaDJvLnNwZWNpZmljaXR5KHJmaDJvX2x1X3BlcmZvcm1hbmNlX25vcm0pWywidG5yIl0sCiAgICAgIHkgPSBoMm8uc2Vuc2l0aXZpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0cHIiXSwKICAgICAgY29sID0gImdyZWVuIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmBgYAoKIyMjIDEzLjQgT3RoZXIgSDJPIE1ldHJpY3MgClNvbWUgYWRkaXRpb25hbCBpbmZvcm1hdGlvbiB0aGF0J3MgZWFzaWx5IG9idGFpbmFibGUgZnJvbSB0aGUgSDJPIGxpYnJhcnkuIAoKYGBge3J9Cmgyby5jb25mdXNpb25NYXRyaXgocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSkKaDJvLmNvbmZ1c2lvbk1hdHJpeChyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZCkKaDJvLmNvbmZ1c2lvbk1hdHJpeChyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pCgpwbG90KGgyby5GMShyZmgyb19zbHQyX3BlcmZvcm1hbmNlKSwgY29sID0gImJsdWUiLCBtYWluID0gIkYxIFBlcmZvcm1hbmNlIikKbGluZXMoaDJvLkYxKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkKSwgY29sID0gInJlZCIpCmxpbmVzKGgyby5GMShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pLCBjb2wgPSAib3JhbmdlIikKCnBsb3QocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZSwgIyBSRU1JTkRFUjogVFBSID0gU2Vuc2l0aXZpdHksIEZQUiA9ICgxIC0gc3BlY2lmaWNpdHkpCiAgICAgdHlwZSA9ICJyb2MiLCAKICAgICBjb2wgPSAicmVkIiwKICAgICBjZXggPSAwLjIsCiAgICAgcGNoID0gMTAKKQoKaDJvLmNvbmZ1c2lvbk1hdHJpeChyZmgyb19sdV9wZXJmb3JtYW5jZSkKcGxvdChoMm8uRjEocmZoMm9fbHVfcGVyZm9ybWFuY2UpKQpwbG90KHJmaDJvX2x1X3BlcmZvcm1hbmNlLAogICAgIHR5cGUgPSAicm9jIiwgCiAgICAgY29sID0gInJlZCIsCiAgICAgY2V4ID0gMC4yLAogICAgIHBjaCA9IDEwCikKYGBgCgojIFYpIFJhbmRvbSBGb3Jlc3QgRXhwbGFpbmVyIChQRFApCgpSYW5kb20gRm9yZXN0IEV4cGxhaW5lciBpcyBhIGdsb2JhbCBpbnRlcnByZXRhYmlsaXR5IG1vZGVsIHRoYXQgdWx0aW1hdGVseSBhbGxvd3MgeW91IHRvIGJldHRlciB1bmRlcnN0YW5kIHlvdXIgUkYgbW9kZWwgYW5kIGNyZWF0ZSB2aXN1YWxseSBhdHRyYWN0aXZlIGFuZCB1c2VmdWwgUERQIHBsb3RzLiAKVGhlIG1ldGhvZG9sb2d5IGZvciBiZWhpbmQgUkZFIGlzIHNpbXBsZTogaWRlbnRpZnkgdGhlIHRvcCBtb3N0IGltcG9ydGFudCBmZWF0dXJlcywgdGhlaXIgaW50ZXJhY3Rpb25zLCBhbmQgdGhlbiBncmFwaCB0aGVzZSBpbnRlcmFjdGlvbnMgaW4gd2hhdCBpcyBlc3NlbnRpYWxseSBhIDJEIFBEUCBwbG90LiAKVW5mb3J0dW5hdGVseSwgd2hlbiB0cnlpbmcgdG8gdXNlIFJGRSB3aXRoaW4gdGhlIG5vdGVib29rLCB0aGUgb3V0cHV0cyBnZW5lcmF0ZWQgZXJyb3JzIGFuZCB0aGUgcGxvdHMgd2VyZSBzaW1wbHkgbm90IHNob3dpbmcuIApUbyBnZXQgYXJvdW5kIHRoaXMgcHJvYmxlbSwgd2Ugd2lsbCBzaW1wbHkgYWRkIHRoZSBncmFwaHMgYXMgZ2VuZXJhdGVkIGJ5IHRoZSBSRkUgbGlicmFyeS4gClRoZSBjb2RlIHRoYXQgZ2VuZXJhdGVkIHRoZXNlIGdyYXBocyBiZSBmb3VuZCBpbiBbaGVyZV0oLi9SRkVfc1JOQS5SKSwgYW5kIHRoZSBvdXRwdXQgaHRtbCBnZW5lcmF0ZWQgYnkgdGhlICpleHBsYWluX2ZvcmVzdCgpKiBmdW5jdGlvbiBjYW4gYWxzbyBiZSBmb3VuZCBbIi4vWW91cl9mb3Jlc3RfZXhwbGFpbmVkLmh0bWwiXSguL1lvdXJfZm9yZXN0X2V4cGxhaW5lZC5odG1sKS4gCktlZXAgaW4gbWluZCB0aGF0IHRoZSBIVE1MIGZpbGUgaXMgYXV0b21hdGljYWxseSBnZW5lcmF0ZWQgYnkgdGhlIGxpYnJhcnkgYW5kIGRvZXMgbm90IGluY2x1ZGUgYWxsIHRoZSBwbG90cyB3ZSBnZW5lcmF0ZWQgKGl0IHNlZW1zIHRvIG9ubHkgc2hvdyB3aGF0IGl0IGNvbnNpZGVycyB0aGUgbWFpbiBwbG90cykuIApTaW5jZSB0aGUgeW91IHN0aWxsIGhhdmUgdG8gZ2VuZXJhdGUgdGhlIHBsb3RzIHlvdXJzZWxmLCB0aGUgSFRNTCBmaWxlIGlzIGEgY29vbCBib251cywgYnV0IG5vdCBhIG5lY2Vzc2FyeS4gCgojIyAxNC4gVmFyaWFibGUgSW1wb3J0YW5jZSBNZWFzdXJlcwojIyMgMTQuMSBEaXN0cmlidXRpb24gb2YgTWluaW1hbCBEZXB0aCBhbmQgaXRzIE1lYW4KIVtEaXN0cmlidXRpb24gb2YgbWluaW1hbCBkZXB0aCBhbmQgaXRzIG1lYW5dKC4vUmVzdWx0cy9yZmVfbWluX2RlcHRoX2Rpc3RyaWJ1dGlvbi5wbmcpCgojIyMgMTQuMiBJbXBvcnRhbmNlIE1lYXN1cmVzIFRhYmxlIApgYGB7cn0KcmZlX2ltcG9ydGFuY2VfZnJhbWUgPC0gcmVhZC5jc3YoIi4vUmVzdWx0cy9SRkVfaW1wb3J0YW5jZV9mcmFtZV9vcmlnUkYuY3N2Iiwgc2VwID0gIiwiLCBoZWFkZXIgPSBUUlVFKQpyZmVfaW1wb3J0YW5jZV9mcmFtZSAKYGBgCgojIyMgMTQuMyBNdWx0aXdheSBJbXBvcnRhbmNlIFBsb3RzCiMjIyMgMTQuMy4xIFRpbWVzIGEgcm9vdCB2cyBNZWFuIG1pbiBkZXB0aCB2cyBOdW1iZXIgb2YgTm9kZXMKIVtUaW1lcyBhIHJvb3QgdnMgTWVhbiBtaW4gZGVwdGggdnMgTnVtYmVyIG9mIE5vZGVzXSguL1Jlc3VsdHMvcmZlX211bHRpd2F5X2ltcF9wbG90XzEucG5nKQoKIyMjIyAxNC4zLjIgQWNjdXJhY3kgRGVjcmVhc2UgdnMuIEdpbmkgRGVjcmVhc2UgdnMuIFRpbWVzIGEgUm9vdAohW0FjY3VyYWN5IERlY3JlYXNlIHZzLiBHaW5pIERlY3JlYXNlIHZzLiBUaW1lcyBhIFJvb3RdKC4vUmVzdWx0cy9yZmVfbXVsdGl3YXlfaW1wX3Bsb3RfMi5wbmcpCiMjIDE1LiBHR3BhaXJzIENvbXBhcmlzb25zClRoZSBmb2xsb3dpbmcgZ3JhcGhzIGFyZSBzaW1wbHkgaGVscGZ1bCBpbiBpZGVudGlmeWluZyBmZWF0dXJlIGltcG9ydGFuY2UKCiFbZ2dwYWlycyBmdWxsXSguL1Jlc3VsdHMvcmZlX2ltcF9nZ3BhaXJzX2Z1bGwucG5nKQoKIVtnZ3BhcmlzIG1haW4gZmVhdHVyZXNdKC4vUmVzdWx0cy9yZmVfaW1wX2dncGFpcnNfbWFpbl9wYWlycy5wbmcpCgohW2dncGFpcnMgaW1wb3J0YW5jZSByYW5raW5nc10oLi9SZXN1bHRzL3JmZV9pbXBfcmFua2luZ3MucG5nKQoKIyMgMTYuIEZlYXR1cmUgSW50ZXJhY3Rpb25zCiMjIyAxNi4xIFRvcCAzMCBGZWF0dXJlIEludGVyYWN0aW9ucwohW1RvcCAzMCBGZWF0dXJlIEludGVyYWN0aW9uc10oLi9SZXN1bHRzL3JmZV9taW5fZGVwdGhfaW50ZXJhY3Rpb25zLnBuZykgCgpGcm9tIHRoaXMgZ3JhcGgsIHdlIGdhdGhlciB0aGF0IHRoZSB0b3AgNSBmZWF0dXJlIGludGVyYWN0aW9ucyBhcmU6CiAgMS4gRG93bkRpc3RhbmNlOlNTCiAgMS4gRGlzdGFuY2U6U1MKICAxLiBEb3duRGlzdGFuY2U6UG9zMTB3cnRzUk5BU3RhcnQsIAogIDEuIERpc3RUZXJtOlNTLCAKICAxLiBEaXN0YW5jZTpQb3MxMHdydHNSTkFTdGFydApFdmVuIHRob3VnaCBEaXN0YW5jZSBhbmQgRG93bkRpc3RhbmNlIHdlcmUgY29uc2lkZXJlZCB0aGUgbW9zdCBpbXBvcnRhbnQgZmVhdHVyZXMsIHRoZWlyIGludGVyYWN0aW9ucyBkaWQgbm90IGFwcGVhciB0byBiZSBhcyBzaWduaWZpY2FudCBhcyB0aGUgb3RoZXJzCgoKCiMjIyAxNi4yIERvd25EaXN0YW5jZSB2cy4gU1MKIVtEb3duRGlzdGFuY2UgdnMuIFNTXSguL1Jlc3VsdHMvcmZlX0Rvd25EaXN0YW5jZV92c19TUy5wbmcpCiogVGhlIGhpZ2hlcihjbG9zZXIgdG8gMCkgdGhlIFNTLCB0aGUgaGlnaGVyIHRoZSBjb25maWRlbmNlIGluIHRoZSBwcmVkaWN0aW9uCgojIyMgMTYuMyBEaXN0YW5jZSB2cy4gU1MKIVtEaXN0YW5jZSB2cy4gU1NdKC4vUmVzdWx0cy9yZmVfRGlzdGFuY2VfdnNfU1MucG5nKQoqIGRpc3RhbmNlcyBncmVhdGVyIHRoYW4gLTg1MCA9PiBoaWdoZXIgY2hhbmNlcyBvZiBoYXZpbmcgYSBib25hIGZpZGUgc1JOQQoqIFNTIGhpZ2hlciB0aGFuIC0xMCA9PiBMb3dlciB0aGUgcHJvYiBvZiBoYXZpbmcgYSBib25hIGZpZGUgc1JOQQoKIyMjIDE2LjQgRG93bkRpc3RhbmNlIHZzLiBQb3MxMHdydHNSTkFTdGFydAohW0Rvd25EaXN0YW5jZSB2cy4gUG9zMTB3cnRzUk5BU3RhcnRdKC4vUmVzdWx0cy9yZmVfRG93bkRpc3RhbmNlX3ZzX1BvczEwd3J0c1JOQVN0YXJ0LnBuZykgCiogTm8gY2xlYXIgZGlzZXJuYWJsZSB0YWtlYXdheQoKIyMjIDE2LjUgRGlzdFRlcm0gdnMuIFNTCiFbRGlzdFRlcm0gdnMuIFNTXSguL1Jlc3VsdHMvcmZlX0Rpc3RUZXJtX3ZzX1NTLnBuZykKKiBTUyBvZiAwIHNlZW1zIHRvIHZpcnR1YWxseSBndWFyYW50ZWUgdGhhdCB3ZSBkb24ndCBoYXZlIGEgYm9uYSBmaWRlIHNSTkEKCiMjIyAxNi42IERpc3RhbmNlIHZzLiBQb3MxMHdydHNSTkFTdGFydAohW0Rpc3RhbmNlIHZzLiBQb3MxMHdydHNSTkFTdGFydF0oLi9SZXN1bHRzL3JmZV9EaXN0YW5jZV92c19Qb3MxMHdydHNSTkFTdGFydC5wbmcpIAoqIEhhdmluZyBhIFBvczEwd3J0c1JOQVN0YXJ0IG5lYXIgMCBvciBzbGlnaHRseSBsb3dlciBzZWVtcyB0byBtYWtlIHByZWRpY3Rpb25zIGZ1enp5IChuZWFyIDAuNSkKCgojIyMgMTYuNyBEaXN0YW5jZSB2cy4gRG93bkRpc3RhbmNlCiFbRGlzdGFuY2UgdnMuIERvd25EaXN0YW5jZV0oLi9SZXN1bHRzL3JmZV9EaXN0YW5jZV92c19Eb3duRGlzdGFuY2UucG5nKSAKKiBUaGVzZSBhcmUgdGhlIDIgdG9wIGZlYXR1cmVzLCBidXQgdGhlaXIgaW50ZXJhY3Rpb25zIGFyZSBub3QgYW1vbmcgdGhlIHRvcCBvbmVzCiogV2hlbiBib3RoIGZlYXR1cmVzIGFyZSBjbG9zZSB0byAwLCB0aGUgY2hhbmNlcyBvZiBoYXZpbmcgYSBib25hIGZpZGUgc1JOQSBncmVhdGx5IGluY3JlYXNlCmRldi5vZmYoKQoKIyMjIDE2LjggRG93bkRpc3RhbmNlIHZzLiBEaXN0VGVybQpCYXNlZCBvbiB0aGUgcHJldmlvdXMgcGxvdHMsIGdlbmVyYXRpbmcgdGhlIGZvbGxvd2luZyBwbG90cyBzZWVtIGxpa2UgYSBzbWFydCB0aGluZyB0byBkbwohW0Rvd25EaXN0YW5jZSB2cy4gRGlzdFRlcm1dKC4vUmVzdWx0cy9yZmVfRG93bkRpc3RhbmNlX3ZzX0Rpc3RUZXJtLnBuZykKKiBoYXZpbmcgYSBEaXN0VGVybSBsZXNzIHRoYW4gNTAsIG9yIGNsb3NlIHRvIDAsIGluY3JlYXNlcyB0aGUgY2hhbmNlcyBvZiBoYXZpbmcgYSBib25hIGZpZGUgc1JOQQoKIyMjIDE2LjkgRGlzdGFuY2UgdnMuIERpc3RUZXJtCiFbRGlzdGFuY2UgdnMuIERpc3RUZXJtXSguL1Jlc3VsdHMvcmZlX0Rpc3RhbmNlX3ZzX0Rpc3RUZXJtLnBuZykKCiogaGF2aW5nIGEgRGlzdGFuY2UgY2xvc2UgdG8gMCBpbmNyZWFzZXMgdGhlIGNoYW5jZXMgb2YgaGF2aW5nIGEgYm9uYSBmaWRlIHNSTkEKCiMjIyAxNi4xMCBzYW1lU3RyYW5kIHZzIHNhbWVEb3duU3RyYW5kCiFbc2FtZVN0cmFuZCB2cyBzYW1lRG93blN0cmFuZF0oLi9SZXN1bHRzL3JmZV9zYW1lU3RyYW5kX3ZzX3NhbWVEb3duU3RyYW5kLnBuZykKCiogc2FtZVN0cmFuZCBhbmQgc2FtZURvd25TdHJhbmQgd2VyZSBjb25zaWRlcmVkIHRoZSBsZWFzdCByZWxldmFudC9pbXBvcnRhbnQgZmVhdHVyZXMuIEhvd2V2ZXIsIHRoZSBvdmVyYWxsIHRvbmUgb2YgdGhlIGdyYXBoIHN1Z2dlc3RzIHRoYXQgaGF2aW5nIHRoZXNlIHZhcmlhYmxlcyBoZWxwcyBib29zdGluZyB0aGUgbW9kZWwncyBjb25maWRlbmNlIGluIGFueSBnaXZlbiBwcmVkaWN0aW9uLiAoaWUuIGNvbXBhcmUgdGhlIHRvbmVzIG9mIHRoZSBwcmV2aW91cyBncmFwaCB0byB0aGlzIG9uZSwgYW5kIGl0J3MgdGhlIG1vZGVsIGhhcyBtb3JlICJjb25maWRlbmNlIiB3aXRoIGl0cyBvdmVyYWxsIGRlY2lzaW9ucyBpbiB0aGlzIGdyYXBoLikKCgojIyAxNy4gUkZFIENvbmNsdXNpb25zClRoZXJlIGFyZSBtYW55IGNvbmNsdXNpb25zIHRoYXQgbWF5IGJlIGdhdGhlcmVkIG9yIG9idGFpbmVkIGZyb20gdGhlIHBsb3RzIGFib3ZlLCBidXQgdGhlIG1haW4gdGFrZWF3YXlzIGNhbiBiZSBzdW1tYXJpemVkIGFzIGZvbGxvd3M6CiAgKiBUaGUgIGNsb3NlciB0aGUgU1MgaXMgdG8gMCwgdGhlIGhpZ2hlciB0aGUgbW9kZWwncyBjb25maWRlbmNlIGluIGl0cyBkZWNpc2lvbgogICogKipIYXZpbmcgYSBEaXN0YW5jZSBiZXR3ZWVuIC04NTAgYW5kIDAgZ3JlYXRseSBpbmNyZWFzZXMgdGhlIGNoYW5jZXMgb2YgaGF2aW5nIGFuIHNSTkEqKgogICogQW4gU1Mgb2YgLTEyIG9yIGhpZ2hlciB3aWxsIGFsbW9zdCBndWFyYW50ZWUgYSBub24gYm9uYSBmaWRlIHNSTkEKICAqIFRoZXJlIGlzIGEgc3dlZXQgc3BvdCBmb3IgZmluZGluZyBib25hIGZpZGUgc1JOQXMgaWYgdGhlIERvd25EaXN0YW5jZSBpcyBsZXNzIHRoYW4gNTAwIGFuZCB0aGUgRGlzdGFuY2UgaXMgbGVzcyB0aGFuIDgwMAogICogKipIYXZpbmcgYSBEb3duRGlzdGFuY2UgbGVzcyB0aGFuIDYwIGFsc28gZ3JlYXRseSBpbmNyZWFzZXMgdGhlIGNoYW5jZXMgb2YgaGF2aW5nIGFuIHNSTkEqKgogICogV2hpbGUgc2FtZVN0cmFuZCBhbmQgc2FtZURvd25TdHJhbmQgd2VyZSBjb25zaWRlcmVkIHRoZSBsZWFzdCBpbXBvcnRhbnQgZmVhdHVyZXMsIHRoZXkgc2VlbSB0byBjb250cmlidXRlIGluIGluY3JlYXNpbmcgdGhlIG1vZGVsJ3MgY29uZmlkZW5jZSAoYXQgbGVhc3QgaW4gY2xhc3NpZnlpbmcgc29tZXRoaW5nIGFzIG5vdCBiZWluZyBhbiBzUk5BLCBhcyB0aGUgZ3JhcGggc2VlbXMgdG8gY29udGFpbiBtb3N0bHkgc29saWQgYmx1ZSBjb2xvcnMpCiAgCipOT1RFL0RJU0NMQUlNRVI6KiBUaGUgbnVtYmVycyBoZXJlIGFyZSBodW1hbiBhcHByb3hpbWF0aW9ucyBiYXNlZCBzb2xlbHkgb24gbG9va2luZyBhdCB0aGUgZ3JhcGhzIGdlbmVyYXRlZCBieSBSRkUsIGFuZCBSRkUgdXNlcyBvbmx5IHRoZSB0cmFpbmluZyBkYXRhIHRvIGdlbmVyYXRlIGl0cyBncmFwaHMuIApJbiBvdGhlciB3b3Jkcywgd2hpbGUgdGhlIGdvYWwgb2YgdGhlIFJGIGlzIHRvIGhhdmUgYSBnZW5lcmFsIG1vZGVsIGFwcGxpY2FibGUgdG8gYW55IHBvc3NpYmxlIGJhY3RlcmlhbCBzUk5BLCB0aGVyZSBpcyBubyBndWFyYW50ZWUgdGhhdCB0aGVzZSBjb25jbHVzaW9ucyBhcHBseSB0byBldmVyeSBzcGVjaWVzIG9mIGJhY3RlcmlhIGluIHRoZSB3b3JsZC4gCgoKIyBWSSkgTElNRQoKKipMSU1FIEZVTkNUSU9OIEFSR1VNRU5UUyoqCgoqIHgJICAgICAgICAgICAgICBUaGUgdHJhaW5pbmcgZGF0YSB1c2VkIGZvciB0cmFpbmluZyB0aGUgbW9kZWwgdGhhdCBzaG91bGQgYmUgZXhwbGFpbmVkLgoqIG1vZGVsCSAgICAgICAgICBUaGUgbW9kZWwgd2hvc2Ugb3V0cHV0IHNob3VsZCBiZSBleHBsYWluZWQKKiBiaW5fY29udGludW91cyAgU2hvdWxkIGNvbnRpbnVvdXMgdmFyaWFibGVzIGJlIGJpbm5lZCB3aGVuIG1ha2luZyB0aGUgZXhwbGFuYXRpb24KKiBuX2JpbnMJICAgICAgICBUaGUgbnVtYmVyIG9mIGJpbnMgZm9yIGNvbnRpbnVvdXMgdmFyaWFibGVzIGlmIGJpbl9jb250aW51b3VzID0gVFJVRQoqIHF1YW50aWxlX2JpbnMJICBTaG91bGQgdGhlIGJpbnMgYmUgYmFzZWQgb24gbl9iaW5zIHF1YW50aWxlcyBvciBzcHJlYWQgZXZlbmx5IG92ZXIgdGhlIHJhbmdlIG9mCiAgICAgICAgICAgICAgICAgICB0aGUgdHJhaW5pbmcgZGF0YQoqIHVzZV9kZW5zaXR5ICAgICBJZiBiaW5fY29udGludW91cyA9IEZBTFNFIHNob3VsZCBjb250aW51b3VzIGRhdGEgYmUgc2FtcGxlZCB1c2luZyBhIGtlcm5lbCBkZW5zaXR5IAogICAgICAgICAgICAgICAgZXN0aW1hdGlvbi4gSWYgbm90LCBjb250aW51b3VzIGZlYXR1cmVzIGFyZSBleHBlY3RlZCB0byBmb2xsb3cgYSBub3JtYWwgZGlzdHJpYnV0aW9uLgoKKipMSU1FIFNFVFRJTkdTIEpVU1RJRklDQVRJT05TKioKCk5PVEU6IG1hbnkgb2YgdGhlc2Ugc2V0dGluZ3Mgd2VyZSBvYnRhaW5lZCB0aHJvdWdoIHRyaWFsIGFuZCBlcnJvciwgYW5kL29yIGJhc2VkIG9uIHRoZSBzdWdnZXN0aW9ucyBwcm92aWRlZCBieSBMSU1FIGFuZCBpdHMgbGlicmFyaWVzLiBTaW5jZSBMSU1FJ3MgcmVzdWx0cyBhcmUgbm90IGFsd2F5cyB0aGUgc2FtZSwgd2UgdXNlZCB0aGUgc2V0dGluZ3MgdGhhdCAgZ2VuZXJhdGVkIHRoZSBtb3N0IGNvbnNpc3RlbnQgYW5kIHZhbGlkIGV4cGxhbmF0aW9ucyBkdXJpbmcgb3VyIGV4cGVyaW5tZW50cy4gT2YgY291cnNlLCB3ZSBlbmNvdXJhZ2Ugb3RoZXJzIHRvIHRyeSBvdXQgb3RoZXIgc2V0dGluZ3MgYW5kIGV4cGVyaW5tZW50IGZ1cnRoZXIuCiogU2V0dGluZ3MgZm9yICJleHBsYWluZXIiCiAgICAqICBiaW5fY29udGludW91cyA9IFRSVUUgLS0+IE1ha2VzIGV4cGxhbmF0aW9uIGVhc2llciwgYW5kIGhhdmluZyB0aGlzIHNldHRpbmcgYXMgdHJ1ZSBoZWxwZWQgaW4gb2J0YWluaW5nIG1vcmUgY29uc2lzdGVudCBhbmQgcmVsaWFibGUgcmVzdWx0cy4KICAgICogIHF1YW50aWxlX2JpbnMgPSBGQUxTRSAtLT4gc2V0dGluZyB0aGlzIHRvIFRSVUUgZ2VuZXJhdGVzIGFuIGVycm9yIGIvYyAgc2FtZVN0cmFuZCBhbmQgc2FtZURvd25TdHJhbmQgZG9uJ3QgaGF2ZSBlbm91Z2ggdmFyaWFuY2UgZm9yIHRoZSBhbGdvcml0aG0gdG8gdXNlIHF1YW50aWxlIGJpbmluZy4gIAogICAgKiAgdXNlX2RlbnNpdHkgPSBUUlVFIC0tPiB3aXRoIGJpbl9jb250aW51b3VzIHNldCB0byBUUlVFLCB0aGlzIHNldHRpbmcgYmVjb21lcyBpcnJlbGV2YW50LiBJbiBzZWN0aW9uIDMuNCwgd2Ugc2hvd2VkIHRoYXQgdGhlIGZlYXR1cmVzIGRvbid0IGZvbGxvdyBhIG5vcm1hbCBkaXN0cmlidXRpb24uIFdlIHRyaWVkIHNldHRpbmcgYmluX2NvbnRpbnVvdXMgPSBGQUxTRSBhbmQgdXNlX2RlbnNpdHkgPSBUUlVFLCBidXQgdGhpcyBnZW5lcmF0ZWQgY29udHJhZGljdG9yeSBleHBsYW5hdGlvbnMgdGhhdCBkaWRuJ3QgbWFrZSBzZW5zZS4gCiAgICAqICBuX2JpbnMgPSAxMCAtLT4gVGhlIGRlZmF1bHQgdmFsdWUgd2Fzbid0IGdpdmluZyB1cyBjb25zaXN0ZW50IHJlc3VsdHMsIHNvIHdlIGJ1bXBlZCB1cCB0aGUgbnVtYmVyIHRvIDEwIGFuZCBzdGFydGVkIGdldHRpbmcgc2VlbWluZ2x5IGdvb2QgcmVzdWx0cwoKKiBTZXR0aW5ncyBmb3IgImV4cGxhaW4iCiAgICAqICBuX2xhYmVscyA9IDEgICAgICAgICAgLS0+IHdlIG9ubHkgaGF2ZSAxIGxhYmVsIAogICAgKiAgbl9mZWF0dXJlcyA9IDcgICAgICAgIC0tPiB3ZSB3YW50ZWQgb3VyIGV4cGxhbmF0aW9ucyB0byBiZSBiYXNlZCBvbiBhbGwgNyBmZWF0dXJlcyAKICAgICogIG5fcGVybXV0YXRpb25zID0gMjAwMCAtLT4gVGhpcyBudW1iZXIgd2FzIHNldCB0aHJvdWdoIHRyaWFsIGFuZCBlcnJvci4gSGF2aW5nIGEgc21hbGwgbnVtYmVyIG9mIHBlcm11dGF0aW9ucyBkb2VzIG5vdCBnZW5lcmF0ZSBjb25zaXN0ZW4gcmVzdWx0cywgc28gYSBoaWdoIG51bWJlciBvZiBwZXJtdXRhdGlvbnMgaXMgcmVjb21tZW5kZWQuIDIwMDAganVzdCBzZWVtZWQgdG8gaGF2ZSB5aWVsZGVkIGdvb2QgcmVzdWx0cywgc28gd2UgZGVjaWRlZCB0byBrZWVwIHRoaXMgdmFsdWUuIElmIHNwZWVkIGlzIGEgY29uY2VybiwgbG93ZXJpbmcgdGhpcyBudW1iZXIgdG8gMTAwMCBzaG91bGQgc3RpbGwgeWllbGQgZ29vZCByZXN1bHRzLiAKICAgICogIGRpc3RfZnVuID0gImdvd2VyIiAgICAtLT4gTElNRSdzIGRlZmF1bHQgZGlzdF9mdW4sIGFuZCBzaW5jZSB3ZSBoYXZlIGNhdGVnb3JpY2FsIGZlYXR1cmVzIChzYW1lU3RyYW5kIGFuZCBzYW1lRG93blN0cmFuZCksIGFjY29yZGluZyB0byBMSU1FJ3MgZG9jdW1lbnRhdGlvbiwgImdvd2VyIiBpcyB0aGUgZnVuY3Rpb25zIG9mIGNob2ljZSB3aGVuIGRlYWxpbmcgd2l0aCBhIG1peHR1cmUgb2YgbnVtZXJpY2FsIGFuZCBjYXRlZ29yaWNhbCBmZWF0dXJlcwogICAgKiAgU2lkZSBOb3RlOiBXZSB0cmllZCBhIGNvdXBsZSBvZiBydW5zIG9mIHRoZSBzY3JpcHQgd2l0aCBldWNsaWRlYW4gZGlzdGFuY2UsIGJ1dCBpdCBkaWRuJ3Qgc2VlbSB0byBoYXZlIG11Y2ggb2YgYW4gaW1wYWN0LiBBZGRpdGlvbmFsbHksIHRoZSBkaWZmZXJlbnQgdHlwZXMgb2Ygc2V0dGluZ3MsIHN1Y2ggYXMgZGlmZmVyZW50IGdvd2VyIHBvd2Vycywga2VybmVsIHdpZHRocywgYW5kIGRpc3RhbmNlIGZ1bmN0aW9ucywgdGhhdCBjYW4gYmUgdHdlYWtlZCBpbiBMSU1FIG1lYW5zIHRoYXQgYSAxMDAlIHRob3JvdWdoIGFuYWx5c2lzIG9mIGV2ZXJ5IHBvc3NpYmlsaXR5IG1heSBiZSBuZWFybHkgaW1wb3NzaWJsZS4gRm9yIHRoaXMgcmVhc29uLCB3ZSB0cmllZCB0byBzdGljayB0byB0aGUgZGVmYXVsdCBzZXR0aW5ncyBhcyBvZmZlcmVkIGJ5IExJTUUuIAoKCiMjIDE4LiBMSU1FIFBSRS1TRVRVUAojIyMgMTguMSBMSU1FIGNvbXBhdGFiaWxpdHkgZnVuY3Rpb24KVGhlIG9yaWdpbmFsIG1vZGVsIHdhcyBidWlsdCB1c2luZyB0aGUgcmFuZG9tRm9yZXN0IGxpYnJhcnkgZm91bmQgaW4gQ1JBTi4gSG93ZXZlciwgZXZlbiB0aG91Z2ggTElNRSBpcyBzdXBwb3NlZCB0byBiZSBtb2RlbCBhZ25vc3RpYywgaXQncyBjdXJyZW50IFIgaW1wbGVtZW50YXRpb24gb25seSBzdXBwb3J0cyBjZXJ0YWluIG1vZGVscyBvdXQgb2YgdGhlIGJveC4gVG8gZXh0ZW5kIHN1cHBvcnQgb2YgdGhlIExJTUUgbGlicmFyeSB0byBvdGhlciBtb2RlbHMsIHRoZSBmb2xsb3dpbmcgZnVuY3Rpb25zIG5lZWQgdG8gYmUgZGVmaW5lZCBmb3IgZWFjaCAgdW5zdXBwb3J0ZWQgbW9kZWwgdHlwZS4gCgpgYGB7cn0KbW9kZWxfdHlwZS5yYW5kb21Gb3Jlc3QgPC0gZnVuY3Rpb24oeCwuLi4pICdjbGFzc2lmaWNhdGlvbicKCnByZWRpY3RfbW9kZWwucmFuZG9tRm9yZXN0IDwtIGZ1bmN0aW9uKHgsIG5ld2RhdGEsIHR5cGUsIC4uLikgewogIHJlcyA8LSBwcmVkaWN0KHgsIG5ld2RhdGEgPSBuZXdkYXRhLCB0eXBlID0gInByb2IiKQogIHN3aXRjaCgKICAgIHR5cGUsCiAgICByYXcgPSBkYXRhLmZyYW1lKFJlc3BvbnNlID0gaWZlbHNlKGFzLnZlY3RvcihyZXNbLDJdKSA+IDAuNSwgIjEiLCAiMCIpLCBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UpLAogICAgcHJvYiA9IGFzLmRhdGEuZnJhbWUocmVzLCBjaGVjay5uYW1lcyA9IEYpIAogICkKfQpgYGAKCiMjIyAxOC4yIERhdGEgdG8gRXhwbGFpbgoqc2FtcGxlRGF0YSwgc2FtcGxlRGF0YV9zY2FsZWQsKiBhbmQgKnNhbXBsZURhdGFfbm9ybSogYWxsIGNvbnRhaW4gdGhlIHNhbWUgZmVhdHVyZXMsIGFuZCBoZW5jZSwgd2Ugc2hvdWxkIGJlIGdldHRpbmcgc2ltaWxhciBleHBsYW5hdGlvbnMgaW4gYWxsIG9mIG91ciByZXN1bHRzLiBXZSBhbHNvIGFwcGx5IExJTUUgdG8gdGhlIHNhbWUgaW5zdGFuY2UgNCB0aW1lcywgd2l0aCB0aGUgcHVycG9zZSBvZiB0ZXN0aW5nIHRoZSB2YWxpZGl0eSBhbmQgY29uc2lzdGVuY3kgb2YgTElNRSdzIGV4cGxhbmF0aW9ucy4gCgpgYGB7cn0Kc2FtcGxlRGF0YSA8LSBzbHQyZGF0YVsxLF0gICMgQSB0cnVlIHNhbXBsZQpzYW1wbGVEYXRhW2MoMjo0KSxdIDwtIHNsdDJkYXRhWzEsXQpzYW1wbGVEYXRhWzUsXSA8LSBzbHQyZGF0YVsxOTg2LF0gIyBBIGZhbHNlIHNhbXBsZQpzYW1wbGVEYXRhW2MoNjo4KSxdIDwtIHNsdDJkYXRhWzE5ODYsXQpzYW1wbGVEYXRhCgpzYW1wbGVEYXRhX3NjYWxlZCA8LSBzbHQyZGF0YV9zY2FsZWRbMSxdICAKc2FtcGxlRGF0YV9zY2FsZWRbYygyOjQpLF0gPC0gc2x0MmRhdGFfc2NhbGVkWzEsXQpzYW1wbGVEYXRhX3NjYWxlZFs1LF0gPC0gc2x0MmRhdGFfc2NhbGVkWzE5ODYsXSAKc2FtcGxlRGF0YV9zY2FsZWRbYyg2OjgpLF0gPC0gc2x0MmRhdGFfc2NhbGVkWzE5ODYsXQpzYW1wbGVEYXRhX3NjYWxlZAoKc2FtcGxlRGF0YV9ub3JtIDwtIHNsdDJkYXRhX25vcm1bMSxdICAKc2FtcGxlRGF0YV9ub3JtW2MoMjo0KSxdIDwtIHNsdDJkYXRhX25vcm1bMSxdCnNhbXBsZURhdGFfbm9ybVs1LF0gPC0gc2x0MmRhdGFfbm9ybVsxOTg2LF0gCnNhbXBsZURhdGFfbm9ybVtjKDY6OCksXSA8LSBzbHQyZGF0YV9ub3JtWzE5ODYsXQpzYW1wbGVEYXRhX25vcm0KYGBgCgojIyAxOS4gTElNRSB3aXRoIFJhbmRvbUZvcmVzdCBMaWJyYXJ5CiMjIyAxOS4xIEluIG9yaWdpbmFsIE1vZGVsIGFzIGlzCldoaWxlIG9uIG9jY2Fzc2lvbiBhbGwgcmVzdWx0cyBtYXkgbG9vayBzaW1pbGFyLCB3ZSByZWNvbW1lbmQgcnVubmluZyB0aGlzIHBhcnRpY3VsYXIgc2VjdGlvbiBhIGZldyB0aW1lcywgYW5kIHlvdSB3aWxsIG5vdGljZSBzaWduaWZpY2FudCBjaGFuZ2VzLCBsaWtlIGZlYXR1cmVzIHRoYXQgbWF5IHN1cHBvcnQgdGhlIGV4cGxhbmF0aW9uIGluIG9uZSBwbG90LCBidXQgY29udHJhZGljdCBpdCBpbiB0aGUgbmV4dC4gCmBgYHtyfQpwcmVkaWN0KG9yaWdSRiwgc2FtcGxlRGF0YVtjKDEsNSksLWMoODo5KV0sIHR5cGUgPSAicHJvYiIpCmxpbWVfZXhwbGFpbmVyX29yaWcgPC0gbGltZShhcy5kYXRhLmZyYW1lKHRyYWluRGF0YVssYygxOjcpXSksIG9yaWdSRiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJpbl9jb250aW51b3VzID0gVFJVRSwgcXVhbnRpbGVfYmlucz1GQUxTRSwgdXNlX2RlbnNpdHkgPSBUUlVFKQpsaW1lX2V4cGxhbmF0aW9uc19vcmlnIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoMTo0KSxjKDE6NyldKSwgbGltZV9leHBsYWluZXJfb3JpZywgbl9sYWJlbHMgPSAxLCBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX29yaWcpCmxpbWVfZXhwbGFuYXRpb25zX29yaWcgPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFbYyg1OjgpLGMoMTo3KV0pLCBsaW1lX2V4cGxhaW5lcl9vcmlnLCBuX2xhYmVscyA9IDEsIG5fZmVhdHVyZXMgPSA3LCBuX3Blcm11dGF0aW9ucyA9IDIwMDApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfb3JpZykKCmBgYAoKIyMjIDE5LjIgSW4gb3JpZ2luYWwgc2NhbGVkIE1vZGVsCgpgYGB7cn0KcHJlZGljdChvcmlnUkZfc2NhbGVkLCBzYW1wbGVEYXRhX3NjYWxlZFtjKDEsNSksLWMoODo5KV0sIHR5cGUgPSAicHJvYiIpCgpsaW1lX2V4cGxhaW5lcl9vcmlnX3NjYWxlZCA8LSBsaW1lKGFzLmRhdGEuZnJhbWUodHJhaW5EYXRhX3NjYWxlZFssIGMoMTo3KV0pLCBvcmlnUkZfc2NhbGVkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJpbl9jb250aW51b3VzID0gVFJVRSwgcXVhbnRpbGVfYmlucz1GQUxTRSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdXNlX2RlbnNpdHkgPSBGQUxTRSwgbl9iaW5zID0gMTApCm5vcm1fZGlzdCA8LSBhcy5kYXRhLmZyYW1lKGxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkJGJpbl9jdXRzKQoKbGltZV9leHBsYWluZXJfb3JpZ19zY2FsZWQgPC0gbGltZShhcy5kYXRhLmZyYW1lKHRyYWluRGF0YV9zY2FsZWRbLCBjKDE6NyldKSwgb3JpZ1JGX3NjYWxlZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnM9RkFMU0UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVzZV9kZW5zaXR5ID0gVFJVRSwgbl9iaW5zID0gMTApCm5vdF9ub3JtX2Rpc3QgPC0gYXMuZGF0YS5mcmFtZShsaW1lX2V4cGxhaW5lcl9vcmlnX3NjYWxlZCRiaW5fY3V0cykKCiMgVGhlIGNvcnJlbGF0aW9ucyBoZXJlIHNob3cgdGhhdCB0aGUgYmlucyB1c2VkIGJ5IHRoZSBleHBsYWluZXIgYXJlIHRoZSBzYW1lIHJlZ2FyZGxlc3Mgb2Ygd2hhdCB2YWx1ZSBpcyBnaXZlbiB0byB0aGUgdXNlX2RlbnNpdHkgcGFyYW1ldGVyLgpjb3Iobm9ybV9kaXN0WywiU1MiXSxub3Rfbm9ybV9kaXN0WywiU1MiXSkKY29yKG5vcm1fZGlzdFssIlBvczEwd3J0c1JOQVN0YXJ0Il0sbm90X25vcm1fZGlzdFssIlBvczEwd3J0c1JOQVN0YXJ0Il0pCmNvcihub3JtX2Rpc3RbLCJEaXN0VGVybSJdLG5vdF9ub3JtX2Rpc3RbLCJEaXN0VGVybSJdKQpjb3Iobm9ybV9kaXN0WywiRGlzdGFuY2UiXSxub3Rfbm9ybV9kaXN0WywiRGlzdGFuY2UiXSkKY29yKG5vcm1fZGlzdFssInNhbWVTdHJhbmQiXSxub3Rfbm9ybV9kaXN0Wywic2FtZVN0cmFuZCJdKQpjb3Iobm9ybV9kaXN0WywiRG93bkRpc3RhbmNlIl0sbm90X25vcm1fZGlzdFssIkRvd25EaXN0YW5jZSJdKQpjb3Iobm9ybV9kaXN0Wywic2FtZURvd25TdHJhbmQiXSxub3Rfbm9ybV9kaXN0Wywic2FtZURvd25TdHJhbmQiXSkKCmxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkJGJpbl9jdXRzICMgQWxsIHRoZSBmZWF0dXJlcyBzZWVtIHRvIGJlIGV2ZW5seSBzcGxpdCBpbnRvIDEwLCBpZSwgYSBkZW5zaXR5IGlzIG5vdCB1c2VkIGZvciB0aGUgYmluIHNwbGl0dGluZy4gIAoKbGltZV9leHBsYW5hdGlvbnNfb3JpZ19zY2FsZWQgPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfc2NhbGVkW2MoMTo0KSxjKDE6NyldKSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGltZV9leHBsYWluZXJfb3JpZ19zY2FsZWQsIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19vcmlnX3NjYWxlZCkKCmxpbWVfZXhwbGFuYXRpb25zX29yaWdfc2NhbGVkIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX3NjYWxlZFtjKDU6OCksYygxOjcpXSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkLCBuX2xhYmVscyA9IDEsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fZmVhdHVyZXMgPSA3LCBuX3Blcm11dGF0aW9ucyA9IDIwMDAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfb3JpZ19zY2FsZWQpCgpgYGAKICAKIyMjIDE5LjMgSW4gb3JpZ2luYWwgbm9ybWFsaXplZCBNb2RlbApgYGB7cn0KcHJlZGljdChvcmlnUkZfbm9ybSwgc2FtcGxlRGF0YV9ub3JtW2MoMSw1KSwtYyg4OjkpXSwgdHlwZSA9ICJwcm9iIikKbGltZV9leHBsYWluZXJfb3JpZ19ub3JtIDwtIGxpbWUoYXMuZGF0YS5mcmFtZSh0cmFpbkRhdGFfbm9ybVssIGMoMTo3KV0pLCBvcmlnUkZfbm9ybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnM9RkFMU0UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVzZV9kZW5zaXR5ID0gVFJVRSwgbl9iaW5zID0gMTApCgpsaW1lX2V4cGxhaW5lcl9vcmlnX25vcm0kYmluX2N1dHMgIyBhbGwgYmlucyBhcmUgc3BsaXQgZXZlbmx5LCBmcm9tIDAgdG8gMSwgaW4gaW5jcmVtZW50cyBvZiAwLjEsIGFzIHdvdWxkIGJlIGV4cGVjdGVkCgpsaW1lX2V4cGxhbmF0aW9uc19vcmlnX25vcm0gPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDE6NCksYygxOjcpXSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX29yaWdfbm9ybSwgbl9sYWJlbHMgPSAxLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKQoKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19vcmlnX25vcm0pCgpsaW1lX2V4cGxhbmF0aW9uc19vcmlnX25vcm0gPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDU6OCksYygxOjcpXSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX29yaWdfbm9ybSwgbl9sYWJlbHMgPSAxLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX29yaWdfbm9ybSkKCgoKYGBgCgojIyAyMC4gTElNRSBpbiBPcmlnIFJGIENvbmNsdXNpb25zCgpXZSB0cmllZCBhcHBseWluZyBMSU1FIHRvIHRoZSBPcmlnIFJGKGluIHBhcnQgMTUpIGhvcGluZyBpdCB3b3VsZCB3b3JrIGxpa2UgYSBwbHVnLWFuZC1wbGF5IGFwcGxpY2F0aW9uLiAKVG8gb3VyIHN1cnByaXNlLCB5b3UgY2FuJ3QgZ2V0IExJTUUgdG8gcHJvcGVybHkgd29yayB3aXRoIHRoZSAiT3JpZ2luYWwiIFJGIG1vZGVsIGJlY2F1c2UgeW91IG11c3Qgbm9ybWFsaXplLCBvciBzY2FsZSB0aGUgZGF0YS4gCkxJTUUgZ2VuZXJhdGVzIHJhbmRvbSBwZXJtdXRhdGlvbnMgdG8gdGhlIGZlYXR1cmVzIGluIG9yZGVyIHRvIGNyZWF0ZSBhIGxpbmVhciBtb2RlbCwgYnV0IGlmIHRoZSBmZWF0dXJlcyBoYXZlIGRpZmZlcmVudCBzY2FsZXMgYW5kIHZhbHVlcywgaXQgaXMgcGxhdXNpYmxlIHRoYXQgZmVhdHVyZXMgd2l0aCBoaWdoZXIgc2NhbGVzIHdpbGwgc2ltcGx5IG91dHdlaWdodCB0aGUgb3RoZXIgZmVhdHVyZXMsIGV2ZW4gaWYgdGhleSdyZSBzaWduaWZpY2FudGx5IGxlc3MgaW1wb3J0YW50LCBhbmQgaGVuY2UgY3JlYXRlIHNrZXdlZCBtb2RlbHMgYW5kIGV4cGxhbmF0aW9ucy4KVG8gdGVzdCB0aGlzIGh5cG90aGVzaXMsIHdlIGNyZWF0ZWQgYSBHTE0gbW9kZWwgYW5kIHJldmlld2VkIGl0cyBSXjIgdmFsdWUsIHdoaWNoIHdhcyBzdXByaXNpbmdseSBsb3csIGFuZCBoZW5jZSwgZmluaXNoZWQgY29udmluY2luZyB1cyB0aGF0IHRoZSBsaW5lYXIgbW9kZWxzIGdlbmVyYXRlZCBieSBMSU1FIHdpdGggdGhlIE9yaWdSRiB3ZXJlIGluYWRlcXVhdGUuIApBZGRpdGlvbmFsbHksIHNpbmNlIExJTUUgcmFuZG9taXplcyB0aGUgcGVybXV0YXRpb24gcG9pbnRzLCBiYXNlZCBvbiBvdXIgZXhwZXJpbm1lbnRzLCBpdCBzZWVtcyB0aGF0IExJTUUgaXMgdXNpbmcgaXRzIG93biByYW5kb21pemF0aW9uIGFsZ29yaXRoIChpZS4gd2UgY291bGRuJ3Qgc2V0IGEgc2VlZCBmb3IgaXQpLCBhbmQgaGVuY2UsIHRoZSBleGFjdCByZXN1bHRzIHdlIG9idGFpbmVkIHdpdGggdGhlIE9yaWdSRiBtYXkgbm90IGJlIGNvbXBsZXRlbHkgcmVwZWF0YWJsZS4gSG93ZXZlciwgZm9yIHRoZSBwYXJ0aWN1bGFyIGNhc2UgaW4gcGFydCAxNSwgdGhhdCBtYXkgYmUgaXJyZWxldmFudCBhcyB0aGUgcG9pbnQgd2UgYXJlIHRyeWluZyB0byBwcm92ZSB0aGVyZSBpcyB0aGF0IExJTUUncyBleHBsYW5hdGlvbnMgd2hlbiB0aGUgZGF0YSBpcyBub3Qgc2NhbGVkIG9yIG5vcm1hbGl6ZWQgYXJlIGluY29uc2lzdGVudCwgYW5kIGhlbmNlIGNhbm5vdCBiZSB0cnVzdGVkLiAgCkZvciBlYWNoIExJTUUgZXhwbGFuYXRpb24gcGxvdCwgc2luY2Ugd2UncmUgaGF2aW5nIExJTUUgZXhwbGFpbiB0aGUgc2FtZSBpbnN0YW5jZSwgYWxsIDQgcGxvdHMgc2hvdWxkIGJlIGlkZW50aWNhbCB0byBlYWNoIG90aGVyLiAKSG93ZXZlciwgd2l0aCB0aGUgT3JpZ1JGIG1vZGVsLCB0aGVyZSdzIHVzdWFsbHkgYXQgbGVhc3QgMSBwbG90IHRoYXQgcmFua3MgdGhlIGZlYXR1cmUgY29udHJpYnV0aW9ucyBkaWZmZXJlbnRseSwgb3IgdGhhdCBtYXkgaGF2ZSBhIGZlYXR1cmUgIlN1cHBvcnRpbmciIGluIG9uZSBwbG90LCBidXQgIkNvbnRyYWRpY3RpbmciIGluIHRoZSBuZXh0LiAKRHVlIHRvIExJTUUncyBuYXR1cmUsIG9idGFpbmluZyBzbGlnaHRseSBkaWZmZXJlbnQgcmVzdWx0cyB3YXMgZXhwZWN0ZWQsIGJ1dCBub3QgdG8gYSBkZWdyZWUgdGhhdCB3b3VsZCBtYWtlIHRoZSBleHBsYW5hdGlvbnMgaW52YWxpZC4gIAoKQnkgc2NhbGluZywgb3Igbm9ybWFsaXppbmcsIHRoZSBkYXRhLCB3ZSB3ZXJlIGFibGUgdG8gb2J0YWluIG1vcmUgY29uc2lzdGVudCByZXN1bHRzIChpZS4gZmVhdHVyZXMgdGhhdCBzdXBwb3J0IG9yIGNvbnRyYWRpY3QgYSBwcmVkaWN0aW9uLCBiZWhhdmUgdGhlIHNhbWUgaW4gYWxsIHRoZSBjb3JyZXNwb25kaW5nIDQgcGxvdHMsIGFsdGhvdWdoIHRoZWlyIGltcG9ydGFuY2UgcmFua2luZyBtYXkgYmUgZGlmZmVyZW50KSwgYW5kIGluIG1vc3QgY2FzZXMsIHRoZSByYW5raW5ncyBvZiBmZWF0dXJlIGNvbnRyaWJ1dGlvbnMgYXMgcGxvdHRlZCBieSBMSU1FIGFjdHVhbGx5IHJlbWFpbmVkIHRoZSBzYW1lLiAKTElNRSBiZWhhdmVzIHByb3Blcmx5IGlmIHRoZSBkYXRhIGlzIHNjYWxlZCBhbmQgbm9ybWFsaXplZC4KCkFuIGFkZGl0aW9uYWwgaW50ZXJlc3RpbmcgdGFrZS1hd2F5LCBpcyB0aGF0IHNhbWVTdHJhbmQgYW5kIHNhbWVEb3duU3RyYW5kIGFyZSBjb25zaWRlcmVkIHRoZSBsZWFzdCBpbXBvcnRhbnQgZmVhdHVyZXMsIGFuZCBzZWVtIHRvIGJlY29tZSBjb21wbGV0ZWx5IGlycmVsZXZhbnQgZm9yIHByZWRpY3Rpb25zIHdoZXJlIHRoZSBpbnN0YW5jZSBpcyBGQUxTRSBhcyBMSU1FIHNpbXBseSBleGNsdWRlIHRoaXMgZmVhdHVyZSAuIAoKCiMjIDIxLiBMSU1FIGZvciB0aGUgSDJPIFJGIG1vZGVscwoKSW5pdGlhbCBzdHJ1Z2dsZXMgd2l0aCBnZXR0aW5nIExJTUUgc2V0IHVwIHdpdGggdGhlIG9yaWdpbmFsIHJhbmRvbSBmb3Jlc3QgbGVhZCB1cyB0byBiZWxpZXZlIHRoYXQgc2luY2UgUmFuZG9tRm9yZXN0IHdhc24ndCBhIHN1cHBvcnRlZCBtZXRob2QgYnkgZGVmYXVsdCBpbiB0aGUgTGltZSBMaWJyYXJ5LCB0aGF0IG1heWJlIHVzaW5nIGEgc3VwcG9ydGVkIG1ldGhvZCB3b3VsZCBoZWxwIHVzIG92ZXJjb21lIHRoZXNlIHByb2JsZW1zLiAKRHVyaW5nIG91ciBpbnZlc3RpZ2F0aW9uLCB3ZSBkaXNjb3ZlcmVkIHRoYXQgSDJPIG9mZmVyZWQgYWRkaXRpb25hbCBJTXMgKFBEUHMgYW5kIFNIQVApLCBhbmQgdGhhdCB0aGVpciBhbGdvcml0aG1zIHdlcmUgbmF0aXZlbHkgc3VwcG9ydGVkIGJ5IHRoZSBMaW1lIGxpYnJhcnkuIApUaGlua2luZyB3ZSBjb3VsZCBraWxsIDMgYmlyZHMgKHRlc3QgMyBJTXMpIHdpdGggb25lIHN0b25lIChvbmUgcGFja2FnZSksIHdlIHByb2NlZWRlZCB0byBjcmVhdGUgdGhlIGVxdWl2YWxlbnQgcHJveHkgbW9kZWxzIGluIEgyTyBhbmQgcnVuIHRoZSBjb3JyZXNwb25kaW5nIHRlc3QuIApUaGUgZmlyc3QgSDJPIFJGIG1vZGVsIHdhcyBjcmVhdGVkIHByaW9yIHRvIHJlYWxpemluZyB3ZSBuZWVkZWQgdG8gaGF2ZSBub3JtYWxpemVkIGFuZCBzY2FsZWQgZGF0YSBzZXRzLCBhcyB0aGVzZSBkYXRhIHByZXByb2Nlc3NpbmcgdGVjaG5pcXVlcyBhcmUgbm9ybWFsbHkgbm90IG5lY2Vzc2FyeSBmb3IgcmFuZG9tIGZvcmVzdHMuIApGdXJ0aGVyIGludmVzdGlnYXRpb25zIGFuZCB0ZXN0aW5nIGFsbG93ZWQgdXMgdG8gZGlzY292ZXIgdGhhdCBub3JtYWxpemluZyBhbmQgc2NhbGluZyB0aGUgZGF0YSBhbGxvd2VkIExJTUUgdG8gZ2VuZXJhdGUgcmVhc29uYWJsZSBleHBsYW5hdGlvbnMgd2l0aCB0aGUgT3JpZ1JGIG1vZGVsLgpFdmVudHVhbGx5IHdlIGRlY2lkZWQgdG8gY3JlYXRlIDUgbW9kZWxzIGluIHRvdGFsLCAyIHdpdGggdGhlIG9yaWdpbmFsIFJhbmRvbUZvcmVzdCBsaWJyYXJ5LCBhbmQgMyB3aXRoIHRoZSBIMk8gbGlicmFyeS4gCkV2ZW4gdGhvdWdoIHRoZSBwZXJmb3JtYW5jZSBtZXRyaWNzIGZvciB0aGUgc2NhbGVkIGFuZCBub3JtYWxpemVkIEgyTyBSRiBtb2RlbHMgZGlkIG5vdCBtYXRjaCB0aG9zZSBvZiB0aGUgT3JpZ1JGLCB3ZSBkZWNpZGVkIHRvIHN0aWxsIGluY2x1ZGUgdGhlbSBpbiB0aGUgZm9sbG93aW5nIHNlY3Rpb25zLiAKCiMjIyAyMS4xIEgyTyBSRiBNb2RlbApgYGB7cn0KaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8oc2FtcGxlRGF0YVtjKDEsOCksYygxOjcpXSkgKQpsaW1lX2V4cGxhaW5lcl9yZmgybyA8LSBsaW1lKCBhcy5kYXRhLmZyYW1lKCB0cmFpbkRhdGFbLGMoMTo3KV0gKSwgcmZoMm8sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJpbl9jb250aW51b3VzID0gVFJVRSwgcXVhbnRpbGVfYmlucyA9IEZBTFNFLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2JpbnMgPSAxMAopCmxpbWVfZXhwbGFuYXRpb25zX3JmaDJvIDwtIGV4cGxhaW4oIGFzLmRhdGEuZnJhbWUoc2FtcGxlRGF0YVtjKDE6NCksYygxOjcpXSApLGxpbWVfZXhwbGFpbmVyX3JmaDJvLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9sYWJlbHMgPSAxLCAgbl9mZWF0dXJlcyA9IDcsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX3Blcm11dGF0aW9ucyA9IDIwMDAgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX3JmaDJvKSAKCmxpbWVfZXhwbGFuYXRpb25zX3JmaDJvIDwtIGV4cGxhaW4oIGFzLmRhdGEuZnJhbWUoc2FtcGxlRGF0YVtjKDU6OCksYygxOjcpXSApLGxpbWVfZXhwbGFpbmVyX3JmaDJvLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9sYWJlbHMgPSAxLCAgbl9mZWF0dXJlcyA9IDcsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX3Blcm11dGF0aW9ucyA9IDIwMDAgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX3JmaDJvKSAKCmBgYAoKIyMjIDIxLjIgSW4gSDJPIFJGIHNjYWxlZCBNb2RlbCAKYGBge3J9Cmgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX3NjYWxlZCwgbmV3ZGF0YSA9IGFzLmgybyhzYW1wbGVEYXRhX3NjYWxlZFtjKDEsOCksYygxOjcpXSkpCmxpbWVfZXhwbGFpbmVyX3JmaDJvX3NjYWxlZCA8LSBsaW1lKCBhcy5kYXRhLmZyYW1lKCB0cmFpbkRhdGFfc2NhbGVkWyxjKDE6NyldICksIHJmaDJvX3NjYWxlZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmluX2NvbnRpbnVvdXMgPSBUUlVFLCBxdWFudGlsZV9iaW5zID0gRkFMU0UpIAoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm9fc2NhbGVkIDwtIGV4cGxhaW4oIGFzLmRhdGEuZnJhbWUoc2FtcGxlRGF0YV9zY2FsZWRbYygxOjQpLGMoMTo3KV0pLCAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX3JmaDJvX3NjYWxlZCwgbl9sYWJlbHMgPSAxLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMCwgZGlzdF9mdW4gPSAiZ293ZXIiICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19zY2FsZWQpCgpsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19zY2FsZWQgPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX3NjYWxlZFtjKDU6OCksYygxOjcpXSksICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGltZV9leHBsYWluZXJfcmZoMm9fc2NhbGVkLCBuX2xhYmVscyA9IDEsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwLCBkaXN0X2Z1biA9ICJnb3dlciIgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX3JmaDJvX3NjYWxlZCkKCmBgYAoKIyMjIDIxLjMgTElNRSB0byBIMk8gUkYgbm9ybWFsaXplZCBNb2RlbCAKYGBge3J9Cmgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX25vcm0sIG5ld2RhdGEgPSBhcy5oMm8oc2FtcGxlRGF0YV9ub3JtW2MoMSw4KSxjKDE6NyldKSApCmxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0gPC0gbGltZSggYXMuZGF0YS5mcmFtZSggdHJhaW5EYXRhX25vcm1bLGMoMTo3KV0gKSwgcmZoMm9fbm9ybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnMgPSBGQUxTRSwgdXNlX2RlbnNpdHkgPSBUUlVFKQoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm9fbm9ybSA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDE6NCksYygxOjcpXSApLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMCwgZGlzdF9mdW4gPSAiZ293ZXIiICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19ub3JtKQoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm9fbm9ybSA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDU6OCksYygxOjcpXSApLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMCwgZGlzdF9mdW4gPSAiZ293ZXIiICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19ub3JtKQoKYGBgCgojIyMgMjEuNCBBcHBseWluZyBMSU1FIHRvIEdMTSBNb2RlbCAoQk9OVVMpIApgYGB7cn0KbGltZV9leHBsYWluZXJfZ2xtaDJvIDwtIGxpbWUoIGFzLmRhdGEuZnJhbWUoIHRyYWluRGF0YVssYygxOjcpXSApLGdsbWgybywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnMgPSBGQUxTRSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1c2VfZGVuc2l0eSA9IEZBTFNFLCBuX2JpbnMgPSAxMCApCmxpbWVfZXhwbGFuYXRpb25zX2dsbWgybyA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFbYygxOjQpLGMoMTo3KV0gKSwgbGltZV9leHBsYWluZXJfZ2xtaDJvLCAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fbGFiZWxzID0gMSwgIG5fZmVhdHVyZXMgPSA3LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX3Blcm11dGF0aW9ucyA9IDIwMDAsIGRpc3RfZnVuID0gImdvd2VyIiAgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX2dsbWgybykKCmxpbWVfZXhwbGFuYXRpb25zX2dsbWgybyA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFbYyg1OjgpLGMoMTo3KV0gKSwgbGltZV9leHBsYWluZXJfZ2xtaDJvLCAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fbGFiZWxzID0gMSwgIG5fZmVhdHVyZXMgPSA3LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX3Blcm11dGF0aW9ucyA9IDIwMDAsIGRpc3RfZnVuID0gImdvd2VyIiAgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX2dsbWgybykKCmBgYAoKCiMjIDI1LiBMSU1FIENPTkNMVVNJT05TIAoKQSBrbm93biBwcm9ibGVtIHdpdGggTElNRSBpcyB0aGF0IGV4cGxhbmF0aW9ucyBtYXkgYmUgaW5jb25zaXN0ZW50IGZyb20gb25lIGluc3RhbmNlIHRvIHRoZSBuZXh0LCBldmVuIHdoZW4gdGhlIGluc3RhbmNlcyBhcmUgdmVyeSBzaW1pbGFyIHRvIGVhY2guIApXaXRoIHRoaXMgaW4gbWluZCwgZWFjaCBpbnN0YW5jZSB3ZSB0ZXN0ZWQgd2FzIHRlc3RlZCBhdCBsZWFzdCA0IHRpbWVzICh3ZSBtYW51YWxseSByYW4gdGhpcyBzZWN0aW9uIG92ZXIgYW5kIG92ZXIgYWdhaW4pICwgYW5kIHdpdGggYSBoaWdoIGVub3VnaCBudW1iZXIgb2YgcGVybXV0YXRpb25zLCB3ZSBtYW5hZ2VkIHRvIGdldCBzb21ld2hhdCBjb25zaXN0ZW50IHJlc3VsdHMuIApIb3dldmVyLCBzb21lIG9mIHRoZSByZXN1bHRzIG9idGFpbmVkIGZyb20gIExJTUUgd2VyZSBoZWF2aWx5IGZsYXdlZCBhcyBzb21ldGltZXMgaXQgd291bGQgeWllbGQgZXhwbGFuYXRpb25zIHRoYXQgY29udHJhZGljdGVkIHRoZSByZXN1bHQuIApGb3Igc29tZSByZWFzb24sIHdoZW4gZXhwbGFpbmluZyBmYWxzZSBpbnN0YW5jZXMsIExJTUUgd291bGQgdGVuZCB0byBsZWF2ZSBvdXQgc2FtZVN0cmFuZCBhbmQgc2FtZURvd25TdHJhbmQgZnJvbSB0aGUgZXhwbGFuYXRpb24uCkFkZGl0aW9uYWxseSwgYSBHTE0gd2FzIGFsc28gdHJhaW5lZCB0byBhdHRlbXB0IHRvIG1lYXN1cmUgdGhlIHZhcmlhbmNlIG9mIHRoZSBkYXRhLCBidXQgdGhlIHJlc3VsdHMgZnJvbSB0aGUgR0xNIHlpZWxkZWQgYW4gUl4yIHZhbHVlIGJldHdlZW4gMC4yIGFuZCAwLjMsIHdpdGggYSBtYXggcmVjYWxsIG9mIDAuMDcuCkluIG90aGVyIHdvcmRzLCB0aGUgR0xNIG1vZGVsIHdhcyBvbmx5IHByZWRpY3RpbmcgZmFsc2UgaW5zdGFuY2VzLCBhbmQgc2luY2UgTElNRSB1c2VzIGxpbmVhciBtb2RlbHMgZm9yIGl0cyBleHBsYW5hdGlvbnMsIGJhc2VkIG9uIG91ciBvYnNlcnZhdGlvbnMsIGl0IGlzIHNhZmUgdG8gY29uY2x1ZGUgdGhhdCBMSU1FIHdhcyBkb2luZyB0aGUgc2FtZS4KV2hlbmV2ZXIgYSBUUlVFIGluc3RhbmNlIHdhcyBhbmFseXplLCBMSU1FJ3MgZXhwbGFuYXRpb25zIHdvdWxkIHNheSB0aGF0IG1vc3QgZmFjdG9ycyBjb250cmFkaWN0IHRoZSBwcmVkaWN0aW9uLCB3aGljaCBpcyBvYnZpb3VzbHkgY291bnRlciBpbnR1aXRpdmUgYW5kIHVzZWxlc3MuCgpEdXJpbmcgdGhlIG1ha2luZyBvZiB0aGlzIHByb2plY3QsIHNldmVyYWwgcHJvYmxlbXMgd2VyZSBlbmNvdW50ZXJlZCB3aGlsZSB0cnlpbmcgdG8gYXBwbHkgTElNRSB0byB0aGUgc1JOQSBSRiBtb2RlbC4gCkxJTUUgYXR0ZW1wdHMgdG8gZXhwbGFpbiBhIGNvbXBsZXggbW9kZWwncyBiZWhhdmlvciBpbiBhIHNtYWxsIHJlZ2lvbiBjb3JyZXNwb25kaW5nIHRvIGEgcGFydGljdWxhciBpbnN0YW5jZSBvZiBpbnRlcmVzdCBieSBhcHBseWluZyBhIGxpbmVhciBtb2RlbC4gCkhvd2V2ZXIsIGluIHRoaXMgcGFydGljdWxhciB1c2UgY2FzZSwgdGhlIG1peHR1cmUgb2YgY2F0ZWdvcmljYWwgYW5kIG51bWVyaWNhbCB2YWx1ZXMsIHBsdXMgdGhlIGRpZmZlcmVuY2VzIGluIHJhbmdlcyB3aXRoaW4gZWFjaCBmZWF0dXJlIHJlc3VsdGVkIGluIHZhcmlvdXMgZXhwZXJpbm1lbnRzIHdpdGhvdXQgY29uc2lzdGVudCByZXN1bHRzLiAKRm9yIHRoaXMgcGFydGljdWxhciB1c2UgY2FzZSwgTElNRSB3YXMgaW1wbGVtZW50IGluIG1vZGVscyB0cmFpbmVkIHdpdGggdGhlIG9yaWdpbmFsIHRyYWluaW5nIGRhdGEsIHNjYWxlZCB0cmFpbmluZyBkYXRhLCBhbmQgbm9ybWFsaXplZCB0cmFpbmluZyBkYXRhLiAKU2V2ZXJhbCBhdHRlbXB0cyB3ZXJlIGFsc28gbWFkZSBhdCBtb2RpZnlpbmcgdGhlIGFyZ3VtZW50cyBmb3IgdGhlIGxpbWUoKSBmdW5jdGlvbiwgc3VjaCBhcyBhbGwgdGhlIHBvc3NpYmxlIHBlcm11dGF0aW9ucyBiZXR3ZWVuIGJpbl9jb250aW51b3VzLCBxdWFudGlsZV9iaW5zIGFuZCB1c2VfZGVuc2l0eS4gCkluIHRoZSBleHBsYWluIGZ1bmN0aW9uLCB0aGUgbnVtYmVyIG9mIHBlcm11dGF0aW9ucyB3YXMgaW5jcmVhc2VkIGZyb20gMTAgdG8gMjAwMCBhbmQgdGhlIEV1Y2xpZGVhbiBhbmQgR293ZXIgZGlzdGFuY2UgZnVuY3Rpb25zIHdlcmUgdGVzdGVkIHdpdGgga2VybmVsX3dpZHRocyByYW5naW5nIGZyb20gMC4wMDEgdG8gNSBiZWNhdXNlIHdlIHdlcmUgbm90IGdldHRpbmcgY29uc2lzdGVudCAKCiMgRikgU0hBUCBWYWx1ZXMgCgpgYGB7cn0KU0hBUF9IMk8xIDwtIGgyby5wcmVkaWN0X2NvbnRyaWJ1dGlvbnMocmZoMm8sIGFzLmgybyhzYW1wbGVEYXRhWyxjKDE6NyldKSkKaGVhZChTSEFQX0gyTzEsMTApCmBgYAoKIyBHKSBQRFAgCgoKYGBge3J9Cmgyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyhzbHQyZGF0YV9oMm8pLCBjb2xzID0gIkRpc3RhbmNlIiwgcGxvdD1UUlVFLCBuYmlucz0yKQoKcGFydGlhbChvcmlnUkYsIHByZWQuZGF0YSA9IHRyYWluRGF0YVssYygxOjcpXSwgeC52YXIgPSAiRGlzdGFuY2UiKQoKcGFydGlhbFBsb3Qob3JpZ1JGLCBwcmVkLmRhdGEgPSB0cmFpbkRhdGFbLGMoMTo3KV0sIHggPSAiRGlzdGFuY2UiICkKCnRyYWluRGF0YVssYygxOjkpXQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAiU1MiKQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAiUG9zMTB3cnRzUk5BU3RhcnQiKQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAiRGlzdFRlcm0iKQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAiRGlzdGFuY2UiKQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAiRG93bkRpc3RhbmNlIikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gInNhbWVTdHJhbmQiKQojaDJvLnBhcnRpYWxQbG90KHJmaDJvLCBkYXRhID0gYXMuaDJvKHRyYWluRGF0YSksIGNvbHMgPSAic2FtZURvd25TdHJhbmQiKQpgYGAKCg==

Interpretability Models in the Random Forest sRNA

Table of Contents